Skip to main content

​
Logs and Metadata

Task‑root folders provide run‑level transparency and support reproducibility. Key locations: logs/, reports/run_reports/, qa/, and ica/.

​
Logs

logs/
└── pipeline.log
  • Single plain‑text log file per task (appends across runs).
  • Contains system messages, warnings, and step‑by‑step notes from the run.
  • Useful for debugging and audit trails.

​
Reports & JSON Metadata

reports/
└── run_reports/
    ├── 128_Rest_EyesOpen_D1004_autoclean_report.pdf
    ├── 128_Rest_EyesOpen_D1004_processing_log.csv
    └── 128_Rest_EyesOpen_D1004_autoclean_metadata.json
  • *_autoclean_metadata.json Machine-readable JSON summary. Captures:
    • Run ID, timestamps, and task name.
    • User/system context (username, host, PID).
    • Directories used (bids/, derivatives/, exports/).
    • Entry point and preprocessing parameters.
    • Status (completed, failed, etc.).
    View Example Metadata JSON
  • *_autoclean_report.pdf (QC report) Human-readable QC report. Includes:
    • Import details (channels, duration, sample rate).
    • Processing details (filters, reference, resampling).
    • Full parameter table for reproducibility.
    View Example QC Report PDF

​
ICA Control Sheet

ica/
└── ica_control_sheet.csv
Tracks ICA components across runs and supports manual review/curation.

​
Example Content

​
Pipeline Execution Log (Example)

The log file captures the complete pipeline execution trace: View Complete Pipeline Log Key pipeline steps documented include: 
2025-09-04 10:19:42 | HEADER   | Starting processing for task: RestingStateTutorial
2025-09-04 10:19:44 | SUCCESS  | Successfully loaded .raw file with all events
2025-09-04 10:21:13 | HEADER   | Resampling data from 1000.0 Hz to 250 Hz...
2025-09-04 10:21:14 | HEADER   | Filtering data...
2025-09-04 10:21:25 | INFO     | Detected 9 bad channels: ['E3', 'E122', 'E107', 'E8', 'E14', 'E56', 'E17', 'E127', 'E2']
2025-09-04 10:21:40 | SUCCESS  | ICA step complete
2025-09-04 10:22:30 | INFO     | Applied ICA, removing components: [0, 1, 2, 4, 7, 8, 12, 13, 17, 18, 20, 28, 30, 31, 32, 35, 38, 45, 50, 64, 70, 98, 115]
2025-09-04 10:22:39 | SUCCESS  | ✓ Task RestingStateTutorial completed successfully

​
Processing Logs

Per‑file CSVs are in reports/run_reports/. A combined task‑level log lives at: 
<task>/preprocessing_log.csv
The CSV provides a comprehensive record of each run: View Complete Processing Log Key metrics captured include:
  • Channel Information: Original (128) vs. post-processing channel counts
  • Bad Channels: ['E56', 'E128', 'E8', 'E14', 'E17', 'E2']
  • Filtering: 1-80 Hz bandpass with 60/120 Hz notch filters
  • Epoching: 29 trials total, 7 rejected (75.8% retained)
  • ICA Components: 16 components removed from 121 total

​
Why These Matter

  • Logs capture the execution trace.
  • Metadata provides both a human-readable report and a machine-readable JSON.
  • Together, they ensure that every run is auditable, reproducible, and shareable.
  • The ICA control sheet will expand this functionality by enabling semi-automatic component review pipelines.