Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
P parabix-devel
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 9
    • Issues 9
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 2
    • Merge requests 2
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • cameron
  • parabix-devel
  • Issues
  • #35

Closed
Open
Created Jan 02, 2023 by cameron@cameronMaintainer

JIT compilation time: CSV tools

The major issue affecting the utility of the Parabix framework and Parabix applications generally is the high cost of JIT compilation.

Consider csv2json. A distinct pipeline is generated for each different "type" of CSV input file. At present, the type of the CSV file is essentially determined by the set of column names. Future developments of the CSV tools are likely to introduce additional attributes to the CSV file "type", such as the type of delimiter (e.g., semicolon or tab instead of comma), data type constraints for each field (e.g., a regular expression constraint for each field), potential options for white space trimming of selected fields, and JSON data type interpretation of each field (generate JSON bool, null, numeric values, rather than strings).

Although caching helps for the second and subsequent run involving any particular CSV file "type", most potential users of these tools will find the high JIT compilation time of the first run unacceptable.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking