CLI Reference

Command-line interface for AutoRAG-Research.

Environment Variables

Variable	Description
POSTGRES_HOST	PostgreSQL host
POSTGRES_PORT	PostgreSQL port
POSTGRES_USER	PostgreSQL user
POSTGRES_PASSWORD	PostgreSQL password

Commands

autorag-research

AutoRAG-Research CLI - RAG research on steroids.

Usage

autorag-research [OPTIONS] COMMAND [ARGS]...

Arguments

No arguments available

Options

Name	Description	Required	Default
-cp, --config-path PATH	Path to configuration directory [env var: AUTORAG_RESEARCH_CONFIG_PATH]	No	-
-V, --version	Show version and exit	No	-
--help	Show this message and exit.	No	-

Commands

Name	Description
health-check	Health check a specific model config.
show	Show available resources.
init	Download default configuration files to...
run	Run experiment pipelines with metrics...
data	Manage PostgreSQL dump files via...
drop	Drop PostgreSQL databases.
ingest	Ingest datasets into PostgreSQL.
plugin	Manage AutoRAG-Research plugins.

Subcommands

autorag-research health-check

Health check a specific model config.

Loads the model config via Hydra and runs a health check to verify the model is functional.

Examples: autorag-research health-check embedding mock autorag-research health-check llm openai-gpt4 autorag-research health-check reranker cohere

Usage

autorag-research health-check [OPTIONS] MODEL_TYPE:{embedding|llm|reranker} NAME

Arguments

Name	Description	Required
MODEL_TYPE:{embedding|llm|reranker}	Model type to check: embedding, llm, or reranker	Yes
NAME	Config name (YAML filename without extension)	Yes

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

autorag-research show

Show available resources.

RESOURCE types: datasets - Available dump files (optionally filter by ingestor name) ingestors - Available data ingestors with their parameters pipelines - Available pipeline configurations metrics - Available evaluation metrics databases - Available PostgreSQL databases

Examples: autorag-research show datasets autorag-research show datasets beir autorag-research show ingestors autorag-research show pipelines autorag-research show metrics autorag-research show databases

Usage

autorag-research show [OPTIONS] RESOURCE:{datasets|ingestors|pipelines|metrics|databases} [NAME]

Arguments

Name	Description	Required
RESOURCE:{datasets|ingestors|pipelines|metrics|databases}	Resource type: datasets, ingestors, pipelines, metrics, or databases	Yes
[NAME]	Resource name (e.g., ingestor name for 'datasets')	No

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

autorag-research init

Download default configuration files to the configured directory.

Downloads configuration files from the AutoRAG-Research GitHub repository to your local configs directory. Existing files are not overwritten.

Examples: autorag-research init autorag-research --config-path=/my/configs init

Usage

autorag-research init [OPTIONS]

Arguments

No arguments available

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

autorag-research run

Run experiment pipelines with metrics evaluation.

Configuration is loaded from configs/experiment.yaml (or specified --config-name).

Examples: autorag-research run --db-name=beir_scifact_test autorag-research run --db-name=beir_scifact_test --verbose autorag-research run --db-name=beir_scifact_test --config-name=my_experiment

Usage

autorag-research run [OPTIONS]

Arguments

No arguments available

Options

Name	Description	Required	Default
-d, --db-name TEXT	Database schema name (required)	No	-
-cn, --config-name TEXT	Config file name without .yaml extension [default: experiment]	No	-
-v, --verbose	Enable verbose logging	No	-
--help	Show this message and exit.	No	-

autorag-research data

Manage PostgreSQL dump files via HuggingFace Hub.

Usage

autorag-research data [OPTIONS] COMMAND [ARGS]...

Arguments

No arguments available

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

Subcommands

autorag-research data restore

Download and restore a dump to PostgreSQL.

Downloads the dump file from HuggingFace Hub (if not cached) and restores it to a PostgreSQL database. The database will be created if it doesn't exist.

Examples: autorag-research data restore beir scifact_openai-small autorag-research data restore beir scifact_openai-small --db-name=my_custom_db autorag-research data restore mteb nfcorpus_bge-small --clean autorag-research data restore beir scifact_openai-small --clean --yes

Usage

autorag-research data restore [OPTIONS] INGESTOR FILENAME

Arguments

Name	Description	Required
INGESTOR	Ingestor name (e.g., beir, mteb)	Yes
FILENAME	Dump filename without .dump extension	Yes

Options

Name	Description	Required	Default
--db-name TEXT	Target database name (defaults to filename)	No	-
--clean	Drop database objects before recreating	No	-
--no-owner / --with-owner	Skip restoration of object ownership [default: no-owner]	No	-
-y, --yes	Skip confirmation prompts	No	-
--help	Show this message and exit.	No	-

autorag-research data dump

Export a database to a dump file.

Creates a PostgreSQL dump file using pg_dump in custom format, which can be restored with 'autorag-research data restore' or pg_restore.

Examples: autorag-research data dump --db-name=beir_scifact_test autorag-research data dump --db-name=beir_scifact_test --output=./backup.dump

Usage

autorag-research data dump [OPTIONS]

Arguments

No arguments available

Options

Name	Description	Required	Default
--db-name TEXT	Database name to dump	Yes	-
-o, --output PATH	Output file path (defaults to .dump)	No	-
--no-owner / --with-owner	Skip output of ownership commands [default: no-owner]	No	-
--help	Show this message and exit.	No	-

autorag-research data upload

Upload a dump file to HuggingFace Hub.

Requires authentication via HF_TOKEN environment variable or 'huggingface-cli login'.

Examples: autorag-research data upload ./scifact.dump beir scifact_openai-small autorag-research data upload ./scifact.dump beir scifact_openai-small -m "Add new dump" autorag-research data upload ./scifact.dump beir scifact_openai-small --repo myorg/custom-repo

Usage

autorag-research data upload [OPTIONS] FILE INGESTOR FILENAME

Arguments

Name	Description	Required
FILE	Path to the dump file to upload	Yes
INGESTOR	Ingestor name (e.g., beir, mteb)	Yes
FILENAME	Target filename without .dump extension	Yes

Options

Name	Description	Required	Default
-r, --repo TEXT	Override HuggingFace repo ID (e.g., myorg/my-repo)	No	-
-m, --message TEXT	Commit message for the upload	No	-
--help	Show this message and exit.	No	-

autorag-research drop

Drop PostgreSQL databases.

Usage

autorag-research drop [OPTIONS] COMMAND [ARGS]...

Arguments

No arguments available

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

Subcommands

autorag-research drop database

Drop a PostgreSQL database.

Examples: autorag-research drop database --db-name=beir_scifact_test autorag-research drop database --db-name=beir_scifact_test --yes

Usage

autorag-research drop database [OPTIONS]

Arguments

No arguments available

Options

Name	Description	Required	Default
--db-name TEXT	Database name to drop	Yes	-
-y, --yes	Skip confirmation prompts	No	-
--help	Show this message and exit.	No	-

autorag-research ingest

Ingest datasets into PostgreSQL.

Usage

autorag-research ingest [OPTIONS] COMMAND [ARGS]...

Arguments

No arguments available

Options

Name	Description	Required	Default
-n, --name TEXT	Ingestor name (beir, mteb, ragbench, etc.). Use 'autorag-research show ingestors' to see all.	No	-
-e, --extra TEXT	Ingestor-specific params as key=value (e.g., --extra dataset-name=scifact)	No	-
--subset [train|dev|test]	Dataset split: train, dev, or test [default: test]	No	-
--query-limit INTEGER	Maximum number of queries to ingest	No	-
--min-corpus-cnt INTEGER	Minimum number of corpus documents to ingest	No	-
--db-name TEXT	Custom database name (auto-generated if not specified)	No	-
--embedding-model TEXT	Embedding model config name from configs/embedding/ [default: openai-small]	No	-
--embed-batch-size INTEGER	Batch size for embedding [default: 128]	No	-
--embed-concurrency INTEGER	Max concurrent embedding calls [default: 16]	No	-
--skip-embedding	Skip embedding step (ingest data only)	No	-
--help	Show this message and exit.	No	-

autorag-research plugin

Manage AutoRAG-Research plugins.

Usage

autorag-research plugin [OPTIONS] COMMAND [ARGS]...

Arguments

No arguments available

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

Subcommands

autorag-research plugin sync

Discover installed plugins and copy their YAML configs into configs/.

Scans all installed packages that register autorag_research.pipelines or autorag_research.metrics entry points. YAML files are copied into the local configs/ directory. Existing files are never overwritten.

Example::

pip install autorag-research-elasticsearch
autorag-research plugin sync

Usage

autorag-research plugin sync [OPTIONS]

Arguments

No arguments available

Options

Name	Description	Required	Default
--help	Show this message and exit.	No	-

autorag-research plugin create

Scaffold a new plugin project in the current directory.

Creates a plugin directory structure with: - pyproject.toml with entry_points configured - Pipeline/metric skeleton code - YAML config file - Basic test file

Example::

autorag-research plugin create my_search --type=retrieval
cd my_search_plugin
pip install -e .
autorag-research plugin sync

Usage

autorag-research plugin create [OPTIONS] NAME

Arguments

Name	Description	Required
NAME	Plugin name (e.g., 'my_custom_retrieval')	Yes

Options

Name	Description	Required	Default
-t, --type TEXT	Plugin type: retrieval, generation, metric_retrieval, metric_generation, ingestor	Yes	-
--help	Show this message and exit.	No	-

Quick Cleanup Example

autorag-research drop database --db-name=beir_scifact_test_openai_small --yes