> ## Documentation Index
> Fetch the complete documentation index at: https://docs.autocleaneeg.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Understanding the Processing Log

> Learn how to interpret the CSV report generated by each pipeline run

Each pipeline run generates a simplified **processing log CSV**. You’ll find:

* Per‑file CSVs under `reports/run_reports/`
* A combined task‑level log at `<task>/preprocessing_log.csv`

These files record the run metadata, preprocessing parameters, and quality‑control results.\
It’s designed to make your analysis **reproducible, transparent, and easy to audit**.

## Processing Log Example Row (Per‑Run CSV)

Below is an example entry from a processing log:

| timestamp           | study\_user | run\_id                    | proc\_state      | subj\_basename             | bids\_subject | task                 | flags                                             | net\_nbchan\_orig | net\_nbchan\_post | proc\_badchans                             | proc\_filt\_lowcutoff | proc\_filt\_highcutoff | proc\_filt\_notch | proc\_filt\_notch\_width | proc\_sRate\_raw | proc\_sRate1 | proc\_xmax\_raw | proc\_xmax\_post | proc\_xmax\_percent | epoch\_length | epoch\_limits | epoch\_trials | epoch\_badtrials | epoch\_percent | proc\_nComps | proc\_removeComps                                            | exclude\_category |
| ------------------- | ----------- | -------------------------- | ---------------- | -------------------------- | ------------- | -------------------- | ------------------------------------------------- | ----------------- | ----------------- | ------------------------------------------ | --------------------- | ---------------------- | ----------------- | ------------------------ | ---------------- | ------------ | --------------- | ---------------- | ------------------- | ------------- | ------------- | ------------- | ---------------- | -------------- | ------------ | ------------------------------------------------------------ | ----------------- |
| 2025-09-03 04:04:03 | unknown     | 01K479NCMBQRRQ8QZTGB2CKWDN | completed\_files | 128\_Rest\_EyesOpen\_D1004 | sub-451559    | RestingStateTutorial | WARNING: 8 potential reference artifacts detected | 128               | 128               | \['E56', 'E128', 'E8', 'E14', 'E17', 'E2'] | 1                     | 80                     | \[60, 120]        | 5                        | 1000.0           | 250.0        | 303.4           | 44.0             | 0.145               | 2.0           | \[-1, 1]      | 29            | 7                | 0.758          | 121          | \[0, 1, 2, 3, 4, 12, 15, 17, 18, 20, 21, 28, 34, 36, 49, 74] |                   |

## Processing Log Column Reference

| Column                       | Example Value                                     | Description                                |
| ---------------------------- | ------------------------------------------------- | ------------------------------------------ |
| **timestamp**                | 2025-09-03 04:04:03                               | Time when the run completed                |
| **study\_user**              | unknown                                           | User ID (or `unknown` if not set)          |
| **run\_id**                  | 01K479NCMBQRRQ8QZTGB2CKWDN                        | Unique identifier for this run             |
| **proc\_state**              | completed\_files                                  | Status of the run                          |
| **subj\_basename**           | 128\_Rest\_EyesOpen\_D1004                        | Original recording file name               |
| **bids\_subject**            | sub-451559                                        | BIDS subject identifier                    |
| **task**                     | RestingStateTutorial                              | Task or condition name                     |
| **flags**                    | WARNING: 8 potential reference artifacts detected | Warnings or notes from preprocessing       |
| **net\_nbchan\_orig**        | 128                                               | Channels in the raw input                  |
| **net\_nbchan\_post**        | 128                                               | Channels after preprocessing               |
| **proc\_badchans**           | \[E56, E128, E8, E14, E17, E2]                    | List of bad channels removed               |
| **proc\_filt\_lowcutoff**    | 1                                                 | High-pass filter cutoff (Hz)               |
| **proc\_filt\_highcutoff**   | 80                                                | Low-pass filter cutoff (Hz)                |
| **proc\_filt\_notch**        | \[60, 120]                                        | Notch filter frequencies applied (Hz)      |
| **proc\_filt\_notch\_width** | 5                                                 | Width of notch filter (Hz)                 |
| **proc\_sRate\_raw**         | 1000.0                                            | Original sampling rate (Hz)                |
| **proc\_sRate1**             | 250.0                                             | Resampled rate (Hz)                        |
| **proc\_xmax\_raw**          | 303.429                                           | Duration of raw recording (s)              |
| **proc\_xmax\_post**         | 44.0                                              | Duration after trimming/cropping (s)       |
| **proc\_xmax\_percent**      | 0.145                                             | Proportion of usable data kept             |
| **epoch\_length**            | 2.0                                               | Epoch length (s)                           |
| **epoch\_limits**            | \[-1, 1]                                          | Time window per epoch (s)                  |
| **epoch\_trials**            | 29                                                | Total epochs created                       |
| **epoch\_badtrials**         | 7                                                 | Rejected epochs                            |
| **epoch\_percent**           | 0.759                                             | Percentage of epochs retained              |
| **proc\_nComps**             | 121                                               | Number of ICA components estimated         |
| **proc\_removeComps**        | \[0,1,2,3,4,12,15,17,18,20,21,28,34,36,49,74]     | Indices of ICA components removed          |
| **exclude\_category**        | —                                                 | Categories of excluded components/channels |

## Why It Matters

* **Traceability**: Every preprocessing decision (filters, bad channels, ICA components) is logged.
* **Reproducibility**: Logs can be used to replicate runs with identical parameters.
* **Quality Control**: Flags highlight potential artifacts or reference issues for review.

## Next Steps

* Check the processing log after each run to review **filter settings, rejected channels, and ICA removals**.
* Use logs to compare across runs (especially when rerunning tasks that generate backups).
* Keep logs with your dataset when sharing for **full reproducibility**.