From paper archives to searchable data

Sagan digitizes archives end to end for agencies and provinces: scanning, print and handwriting recognition, metadata, checks, storage and search, in parallel at millions of pages a day.

Book a survey See the process
Paper Scan Recognition, metadata Index

  • 99.98%

    of data fields correct, measured on each batch's acceptance sample

  • 2M+

    pages recognised a day on one cluster; scanning grows with stations

  • 300 dpi

    standard scan resolution, PDF/A with a searchable text layer

  • 100%

    of pages have their image checked as they are scanned

Process

Twelve steps, one closed loop

Paper leaves the store, is digitized and goes back to its place; every page leaves a trace at each step.

AIntake

  1. 01
    Survey the archiveVolume, record types, paper condition, sizes and classification.
  2. 02
    Plan in batchesBatches by store, fonds and record type, with targets and priorities.
  3. 03
    Barcoded handoverA code for every box and file; a signed handover with sheet counts; paper always tracked.
  4. 04
    Prepare the paperUnstaple, flatten, sort by size, insert separators; damaged items listed for repair first.

BDigitize

  1. 05
    Scan to standard300 dpi, colour or greyscale by type; a colour target checked daily; every image checked as it is scanned.
  2. 06
    Clean the imageDeskew, crop, despeckle, split spreads, drop blanks; the uncompressed master is kept.
  3. 07
    Recognise and extractLayout, print, handwriting, tables and seals; metadata per record template.
  4. 08
    Check and acceptLow-confidence fields go to reviewers; every batch is accepted on a random sample.

CHand over

  1. 09
    Package and storePDF/A with text, structured metadata, integrity hashes, several copies.
  2. 10
    Load the databaseMetadata linked to every file, loaded into archive software or sector databases.
  3. 11
    Return the paperCounted, reordered, sealed and handed back to the store against a signature.
  4. 12
    Search and useFull-text and metadata search, cited answers with Genie.

Recognition

Reads print and handwriting alike

The model reads a page the way a person does: layout, tables and seals, then the right fields for each record type.

Extracted metadata

TypeDecision99.9%

Reference125/QD-UBND99.7%

Issued12/03/200899.8%

IssuerDistrict A People's Committee99.6%

SubjectResidential land allocation98.9%

Seal and signature: detected

Handwriting recognised

Full nameNguyễn Thị Lan99.4%

Date of birth14/02/198799.1%

Place of birthXã B, huyện A86.2%Confirmed by a reviewer

FatherNguyễn Văn Hùng98.7%

MotherLê Thị Mai98.9%

  • Print
  • Handwriting
  • Tables
  • Seals and signatures
  • Old, faded, stained paper
  • Large formats up to A0
  • Multi-column, multi-page
  • Vietnamese diacritics

Accuracy

99.98% through four layers of control

The share of correct data fields, measured on each batch's random acceptance sample after human review. A batch that falls short is redone in full.

  1. 1
    Image normalisationStraight, clean, well-lit images before reading.
  2. 2
    Current-generation recognitionVision-language models read the whole page like a person and score each character and field.
  3. 3
    Rules and reference listsDates, reference numbers, places and agencies are checked against lists and business rules.
  4. 4
    Reviewers and acceptanceOnly low-confidence fields reach a person; every batch is accepted on a random sample.
95%96%97%98%99%100%96%Layer 198.6%Layer 299.6%Layer 399.98%Layer 4
2wrong fields in 10,000, after layer 4

Capacity

Millions of pages a day, in parallel

Every page is an independent job. Stations push images to one queue and the recognition cluster grows with the scanning rate, so more machines means more output.

Stations Queue Recognition cluster Review Archive Stations Queue Recognition cluster Review
  • Proof before signingA one-day run on the agency's own files, watched by its staff; figures exported from the system log.
  • Weekly output commitmentsWeekly output and accuracy are written into the contract and reported on the dashboard.
  • Scanning grows with stationsRecognition reaches 2 million pages a day per cluster; scanning output grows with stations and shifts.

Estimate your project

Drag to change; results update at once.

1,03 millionpages scanned a day

10days of scanning

420people on site

5recognition servers

≈ 2 monthsin all, with survey, pilot, set-up and acceptance

Rates used: 2,400 pages an hour for loose paper, 600 for bound volumes, 8-hour shifts at 85% efficiency; each station and shift needs 2 people preparing, 1 scanning and half a reviewer; one recognition server handles 250,000 pages a day; 3 weeks of survey, 4 of pilot, 2 of set-up. Real figures are fixed after the survey.

Equipment

The right machine for every kind of paper

Every scanner runs the same software: files named by barcode, images checked at once, all sent to one recognition queue.

  • High-speed scannersA4 and A3, automatic feed, both sides in one pass, 100 to 200 pages a minute.
  • Contactless book scannersFor bound registers and fragile paper: captured from above, spines left intact.
  • Large-format scannersMaps, drawings and cadastral plans up to A0.
  • Review stationsDual screens: the image beside the recognised data, corrected and approved by batch.
  • One capture softwareOn every scanner, through TWAIN and ISIS.
  • Named by barcodeFiles carry box, file and sheet codes.
  • Instant image checkBlur, skew, missing pages and double feeds stopped at once.
  • One queueEvery image enters the same recognition line.

Old and fragile records

  • Condition triage before scanning; torn or decayed items listed for repair
  • Contactless book scanning on a V cradle, spines left intact
  • LED light without UV or heat
  • Gloves and a separate area for valuable records

The project team

  • A steering board, an on-site technical team, a lead for every line
  • Local hiring and training first
  • Background checks and confidentiality agreements before entering the store
  • Process training and a test before the first shift

Storage and search

Find a record in seconds

Type with or without diacritics, by content or by reference number; ask Genie for an answer with the original page.

  • PDF/A with a text layerLong-term preservation, every word searchable.
  • Structured metadataFields per record type, exported as XML and JSON.
  • IntegrityA hash for every file, copies in separate places.
  • IntegrationAPIs to archive software, service portals and sector databases.

land allocation 2008

3 results, 0.4 s

  1. Decision 125/QD-UBND on residential land allocation

    2008District ADecision

  2. Decision 98/QD-UBND, land allocation for the commune B primary school

    2008District ADecision

  3. Proposal for land allocation, hamlet C residential area

    2008Resources officeProposal

Security

Safe from the sheet of paper to the data file

Built to the graded information-security requirements (TCVN 11930:2017); Sagan prepares the security-level dossier with the agency.

  • A controlled scanning roomAccess cards, cameras on every shift, no phones or USB drives near the records.
  • Paper always accounted forEvery box and file is barcoded, signed for and located at all times.
  • Data stays with the agencyRecognition models run on servers in the agency's own data centre; nothing goes through an outside service.
  • Encryption and rolesEncrypted at rest and in transit; each role sees only its own work.
  • Personal data protectionPersonal details masked from reviewers who do not need them, in line with personal data protection rules.
  • Classified recordsA separate air-gapped line, run or supervised by staff the agency appoints under the State Secrets Protection Law; kept out of the shared index, with the agency holding the drives.

Acceptance and handover

Measurable, checkable, and yours

Batch acceptance

  • Sample size per batch by ISO 2859-1
  • Page count matches the sheets on the handover record
  • Page order, resolution and metadata all correct
  • An acceptance report signed by the agency's board

Output standards

  • PDF/A preservation copies with text (ISO 19005)
  • Lossless TIFF masters
  • Metadata to the Ministry of Home Affairs electronic archive standard (Circular 02/2019/TT-BNV)
  • A file manifest with SHA-256 hashes

The data is the agency's

  • Every image, metadata record and index belongs to the agency
  • Open formats, no dependence on Sagan software
  • Sagan's working copies wiped, with a signed record
  • The search system handed over, users trained

Carried out under the 2024 Archives Law, the clerical work rules (Decree 30/2020/ND-CP) and the State Secrets Protection Law.

Delivery

Run as a provincial programme

One steering board, an on-site technical team, and progress by store and batch on a live dashboard.

  1. 01
    Survey2 to 4 weeksInventory, metadata templates, site and staffing plan.
  2. 02
    Pilot1 store, 4 weeksThe full process on one store, measuring real throughput and accuracy.
  3. 03
    Full rolloutTo planMany stations and stores in parallel, batches accepted every week.
  4. 04
    HandoverOngoingSearch system handed over, training, warranty and ongoing digitization of new records.

Programme dashboardIllustrative figures

  • Scanned68%
  • Recognised66%
  • Accepted61%
  • Accuracy99.98%

Pages a day, last 14 days

  • Store A, land recordsScanning
  • Store B, civil registersChecking
  • Store C, administrative papersAccepted

Start with a survey of your archive

A trial run on your own records before you decide.

Book a survey Search the archive with Genie