# BoM Primary Source Research Viewer

Local interactive research table for primary source documentary work — modeled on [Mormonr.org](https://mormonr.org)'s research table architecture.

## Quick Start

```bash
cd ~/Downloads/Hermes/bom-research-viewer
python3 -m http.server 8080
# Open http://localhost:8080
```

The viewer defaults to `data/witness_accounts.json` (109 BoM translation witness accounts).
To load other datasets, use the `?data=` URL parameter:

| URL | Dataset | Sources |
|-----|---------|---------|
| http://localhost:8080 | BoM Translation Witness Accounts | 109 |
| http://localhost:8080?data=data/sources_full.json | All Translation Methods sources | 238 |
| http://localhost:8080?data=data/sources.json | Original 15-source sample | 15 |

## Project Structure

```
bom-research-viewer/
├── index.html                    # Single-page viewer (no dependencies)
├── README.md                     # This file
├── PIPELINE.md                   # ★ Four-facet research pipeline (ID → Acquire → Organize → Display)
├── data/
│   ├── witness_accounts.json     # 109 witness accounts (default)
│   ├── sources_full.json         # All 238 Mormonr translation sources
│   ├── sources.json              # Original 15-source sample
│   ├── hearsay-taxonomy.md       # 35-tag hearsay classification reference
│   └── images/                   # Downloaded page images per source
│       ├── src-009/
│       └── ...
└── scripts/
    ├── add_source.py             # Add sources (interactive or batch import)
    ├── new_page.py               # Create a new blank research page
    ├── seed_images.py            # Classify sources & seed image fields
    ├── download_images.py        # Download page images (IA, JSP, contentDM, etc.)
    ├── audit.py                  # Quality check (required fields, dead links, etc.)
    └── deploy.py                 # Upload data + images to VPS
```

## Complete Pipeline

See **PIPELINE.md** for the full four-facet process. Quick reference:

```bash
# 1. Create a new page
python3 scripts/new_page.py "Topic Name"

# 2. Add sources (after extracting from Mormonr or manual research)
python3 scripts/add_source.py --data topic.json --batch extracted.json

# 3. Classify & download images
python3 scripts/seed_images.py --data topic.json
python3 scripts/download_images.py --data topic.json --type internet_archive --pages 1

# 4. Audit quality
python3 scripts/audit.py --data topic.json

# 5. Deploy to VPS
python3 scripts/deploy.py --data topic.json
```

## Features

- **Sortable table** — click column headers (Date, Summary, Source, Hearsay)
- **Expandable rows** — full metadata, bibliographic citation, Internet Archive link, verbatim transcription
- **Image gallery** — downloaded page images with navigation, thumbnails, and full-size view
- **Hearsay taxonomy** — 35 classification tags with color-coded badges
- **Full-text search** — searches all fields including transcriptions (`Cmd+F`)
- **Filter by hearsay type or tag** — dropdowns
- **Permalinks** — URL hash encodes view state; shareable links
- **Expand/Collapse All** — `Cmd+E` / `Cmd+W`
- **Print-friendly** — `Cmd+P`
- **Multi-dataset** — `?data=` URL parameter loads any JSON file in `data/`
- **Deployed** — http://bomresearcher.fit

## Data Schema

Each source record:

```json
{
  "id": "unique-id",
  "date": "Aug 11, 1829",
  "date_sort": 1829.595,
  "summary": "One-line description",
  "source": "Person or entity",
  "hearsay": "Direct Journalism",
  "type": "Newspaper",
  "reference": "Full Chicago citation",
  "source_link": "https://archive.org/...",
  "scribe_publisher": "Who recorded it",
  "people": ["Person A", "Person B"],
  "audience": "Reading Public",
  "transcription": "Full text...",
  "notes": "Context or significance",
  "tags": ["keyword", "tags"]
}
```

See the full skill documentation: `skill_view('primary-source-research-viewer')`

## Adding Sources

```bash
# Interactive prompts
python3 scripts/add_source.py

# Target a specific dataset
python3 scripts/add_source.py --data witness_accounts.json

# Import from JSON file
python3 scripts/add_source.py --file my_source.json

# Batch import
python3 scripts/add_source.py --batch new_sources.json
```

## Creating New Research Pages

```bash
# Create empty dataset for a new topic
python3 scripts/new_page.py "Book of Mormon Witnesses"

# Creates data/book-of-mormon-witnesses.json
# View at http://localhost:8080?data=data/book-of-mormon-witnesses.json
```

## Extracting from Mormonr

1. Navigate to any Mormonr Q&A research table
2. Click **EXPAND ALL**
3. Paste the JS extraction script in browser console (see `primary-source-research-viewer` skill → references)
4. Process with Python to parse detail text into structured fields
5. Save to `data/<topic>.json`

Full guide: `skill_view('primary-source-research-viewer', file_path='references/mormonr-extraction.md')`

## Current Datasets

### witness_accounts.json (109 sources)
First and second-hand witness accounts describing the Book of Mormon translation process. Filtered from Mormonr's Translation Methods and Seer Stones pages. Key witnesses: Joseph Smith (33), David Whitmer (12), Oliver Cowdery (6), William McLellin (5), Emma Smith (3), Martin Harris (3), Lucy Mack Smith (3), plus Philadelphia Album, Benton/Cowdery sworn testimony, and 8 others.

- **94% source link coverage** (103 of 109 have direct links to original documents)
- **100% have full bibliographic references and transcriptions**
- **5 physical-archive-only sources** documented with repository information

### sources_full.json (238 sources)
Complete extraction from Mormonr's "Book of Mormon Translation Methods" page. All 238 primary sources, 600 BC – 2021.

### sources.json (15 sources)
Original hand-curated sample used for initial viewer development.