Workflow

[petermr]

automation of tasks and workflow

[petermr]

Workflow for unsupervised

Query

• pygetpapers -> directory
• to EPMC
• 100-500 articles
• software (pygetpapers/v20)
• regular testing

corpus

• make HTML
  • from PDF
  • from XML
• make sections
• metadata
• annotate documents (optional) amilib
• software
  • amilib/pdf2html (needs revisiting)
  • amilib/xml2html
  • xpath for sections (section dictionary)

extract keyphrases

• use corpus
• select sections
• docanalysis OR txt2phrases (merge?)
• choose/allow dictionaries?
  • countries
  • organizations
    o UN
• normalize case
• CSV file:
  • phrase
  • count
  • list of documents/ pointers
• software
  • move to python 3.12
  • add tests
  • manually review functionality

manual examination

• false positive removal / annotate (CSV)
• revise query?

encyclopedia

• flexible input
  • many files or 1
  • CSV or TXT
• removal of false positives
• definition/description text (original or Wikipedia)
• disambiguation
  • manual
• sorting
  • alphabetical
  • weighted by frequency

knowledge graph

• nodes (with tooltip)
  • weight
  • url
  • definition
• edges/links
  • similarity by description text
  • seeAlso
  • parent - child