rommapp_romm/docs/BACKEND_ARCHITECTURE.md

RomM Backend Architecture

Comprehensive documentation of the RomM backend: a FastAPI-based server powering the self-hosted retro gaming platform.

---

Table of Contents

1. Overview
2. High-Level Architecture
3. Directory Structure
4. Application Lifecycle
5. Database Layer
6. API Endpoints
7. Authentication & Authorization
8. Business Logic (Handlers)
9. External Integrations (Adapters)
10. Real-Time Communication (WebSockets)
11. Background Tasks & Scheduling
12. File System Management
13. Caching (Redis)
14. Configuration
15. Error Handling
16. Logging
17. Testing

---

1. Overview

Property	Value
Framework	FastAPI 0.121.1
Language	Python 3.13+
ORM	SQLAlchemy 2.0
Migrations	Alembic
Databases	MariaDB, MySQL, PostgreSQL
Cache/Queue	Redis (via RQ)
Real-time	Socket.IO (python-socketio)
Auth	OAuth2 + Basic + OIDC + Sessions
ASGI Server	Uvicorn / Gunicorn
Error Tracking	Sentry

RomM's backend is responsible for:

• Library scanning — detecting platforms and ROMs from the filesystem
• Metadata enrichment — pulling game info from 10+ external providers
• User management — roles, authentication, per-user game tracking
• Asset management — saves, save states, screenshots, firmware/BIOS
• Device sync — cross-device save synchronization
• Netplay — real-time multiplayer room coordination
• Feed generation — Tinfoil, WebRcade, PKGi, and other custom formats

---

2. High-Level Architecture

                                 +---------------------+
                                 |     Nginx / CDN     |
                                 |  (reverse proxy,    |
                                 |   X-Accel-Redirect) |
                                 +----------+----------+
                                            |
                                            v
                              +-------------+--------------+
                              |     FastAPI Application     |
                              |        (main.py)            |
                              +-------------+--------------+
                                            |
              +-----------------------------+------------------------------+
              |                             |                              |
     +--------v--------+         +---------v----------+         +---------v--------+
     |   Middleware     |         |   API Routers      |         |   WebSockets     |
     |  Stack (5 layers)|         |  (20 routers)      |         |  /ws  /netplay   |
     +---------+--------+         +---------+----------+         +---------+--------+
               |                            |                              |
               v                            v                              v
     +-------------------+      +-----------+-----------+       +----------+---------+
     | CORS              |      |    Endpoint Layer     |       |  Socket.IO Server  |
     | CSRF              |      | (request validation,  |       |  (scan progress,   |
     | Authentication    |      |  response schemas)    |       |   netplay rooms)   |
     | Session (Redis)   |      +-----------+-----------+       +----------+---------+
     | Context Vars      |                  |                              |
     +-------------------+                  v                              |
                              +-------------+--------------+               |
                              |     Handler Layer          |               |
                              | (business logic, CRUD,     |<--------------+
                              |  metadata, filesystem)     |
                              +------+-------+------+------+
                                     |       |      |
                  +------------------+       |      +------------------+
                  |                          |                         |
         +--------v--------+     +----------v----------+    +---------v---------+
         |   Database       |     |   External APIs     |    |   File System     |
         |  (SQLAlchemy)    |     |  (IGDB, MobyGames,  |    |  (ROM library,    |
         |                  |     |   ScreenScraper,     |    |   assets, BIOS)   |
         | MariaDB/MySQL/PG |     |   SteamGridDB, RA,   |    |                   |
         +---------+--------+     |   LaunchBox, HLTB,   |    +-------------------+
                   |              |   Hasheous, TGDB,    |
                   v              |   Flashpoint)        |
         +-------------------+    +---------------------+
         |     Redis          |
         | (sessions, cache,  |
         |  job queues, rooms)|
         +-------------------+

Layered Architecture

+-----------------------------------------------------------------------+
|                         PRESENTATION LAYER                             |
|  endpoints/          API route handlers, request/response schemas      |
|  endpoints/sockets/  WebSocket event handlers                          |
+-----------------------------------------------------------------------+
|                         BUSINESS LOGIC LAYER                           |
|  handler/auth/       Authentication & authorization                    |
|  handler/database/   CRUD operations per entity                        |
|  handler/metadata/   Metadata fetching & normalization                 |
|  handler/filesystem/ File system operations                            |
|  handler/            Scan orchestration, netplay, socket management    |
+-----------------------------------------------------------------------+
|                         DATA ACCESS LAYER                              |
|  models/             SQLAlchemy ORM model definitions                  |
|  adapters/services/  External API client wrappers                      |
|  handler/redis_handler.py  Cache & queue operations                    |
+-----------------------------------------------------------------------+
|                         INFRASTRUCTURE LAYER                           |
|  config/             Environment variables & YAML config               |
|  decorators/         Auth & DB session decorators                      |
|  exceptions/         Custom exception hierarchy                        |
|  logger/             Structured logging                                |
|  utils/              Shared helpers (hashing, context, validation)     |
|  tasks/              Background job definitions & scheduling           |
+-----------------------------------------------------------------------+

---

3. Directory Structure

backend/
├── main.py                    # FastAPI app creation, middleware, routers
├── startup.py                 # Pre-startup: cache init, scheduled tasks
├── watcher.py                 # Filesystem change detection (watchfiles)
├── __version__.py             # Version placeholder
├── alembic.ini                # Alembic migration configuration
├── pytest.ini                 # Test configuration
├── .coveragerc                # Code coverage settings
│
├── adapters/                  # External API client wrappers
│   └── services/
│       ├── igdb.py            # IGDB (Internet Game Database)
│       ├── mobygames.py       # MobyGames
│       ├── screenscraper.py   # ScreenScraper
│       ├── steamgriddb.py     # SteamGridDB
│       ├── retroachievements.py # RetroAchievements
│       ├── rahasher.py        # RA hash computation
│       ├── libretro_thumbnails.py # Libretro thumbnails (covers/screenshots)
│       └── *_types.py         # Type definitions per adapter
│
├── alembic/                   # Database migrations
│   ├── env.py                 # Migration environment setup
│   └── versions/              # 80+ migration scripts
│
├── config/                    # Configuration system
│   ├── __init__.py            # Env var loading (100+ variables)
│   └── config_manager.py      # YAML config manager (singleton)
│
├── decorators/                # Function decorators
│   ├── auth.py                # @protected_route, OAuth setup
│   └── database.py            # @begin_session (DB session injection)
│
├── endpoints/                 # API route handlers
│   ├── auth.py                # Login, logout, token, OIDC
│   ├── user.py                # User CRUD, invite links
│   ├── client_tokens.py       # API token management
│   ├── platform.py            # Platform CRUD
│   ├── collections.py         # Collection management
│   ├── configs.py             # App configuration
│   ├── device.py              # Device registration
│   ├── export.py              # ES-DE gamelist.xml + Pegasus exports
│   ├── feeds.py               # Tinfoil, WebRcade, PKGi feeds
│   ├── firmware.py            # BIOS/firmware management
│   ├── heartbeat.py           # Health check + setup wizard
│   ├── netplay.py             # Netplay room listing
│   ├── play_sessions.py       # Play session ingestion & queries
│   ├── raw.py                 # Raw asset file serving
│   ├── saves.py               # Save file management
│   ├── screenshots.py         # Screenshot management
│   ├── search.py              # Cross-provider metadata search
│   ├── states.py              # Save state management
│   ├── stats.py               # Library statistics
│   ├── sync.py                # Device sync sessions (push/pull, SSH)
│   ├── tasks.py               # Task monitoring & triggering
│   ├── roms/                  # ROM-specific endpoints
│   │   ├── __init__.py        # ROM CRUD, download, bulk ops
│   │   ├── upload.py          # Chunked upload system
│   │   ├── files.py           # File download (nginx redirect)
│   │   ├── manual.py          # Manual metadata entry
│   │   └── notes.py           # ROM notes/comments
│   ├── sockets/               # WebSocket handlers
│   │   ├── scan.py            # Scan progress events
│   │   └── netplay.py         # Netplay room events
│   ├── forms/                 # Request body models
│   │   └── identity.py        # Auth form schemas
│   └── responses/             # Pydantic response schemas
│       ├── base.py            # Base response classes
│       ├── rom.py             # SimpleRomSchema, DetailedRomSchema
│       ├── platform.py        # PlatformSchema
│       ├── identity.py        # UserSchema, TokenResponse
│       └── ...                # 15+ more response schemas
│
├── exceptions/                # Custom exception classes
│   ├── auth_exceptions.py     # AuthCredentialsException, etc.
│   ├── endpoint_exceptions.py # NotFound, Permission, Conflict
│   ├── fs_exceptions.py       # Filesystem errors
│   ├── config_exceptions.py   # Config write errors
│   ├── task_exceptions.py     # Scheduler errors
│   └── socket_exceptions.py   # Scan stopped
│
├── handler/                   # Business logic layer
│   ├── scan_handler.py        # Library scan orchestration
│   ├── socket_handler.py      # Socket.IO server management
│   ├── netplay_handler.py     # Netplay room state
│   ├── redis_handler.py       # Redis clients & queues
│   ├── auth/                  # Authentication subsystem
│   │   ├── base_handler.py    # Auth, OAuth, OIDC handlers
│   │   ├── hybrid_auth.py     # Multi-method auth backend
│   │   ├── constants.py       # Scopes, role mappings
│   │   └── middleware/        # CSRF, session middleware
│   ├── database/              # Per-entity CRUD handlers
│   │   ├── base_handler.py    # Engine, session factory
│   │   ├── roms_handler.py    # ROM queries & mutations
│   │   ├── platforms_handler.py     # Platform CRUD, slug mapping
│   │   ├── users_handler.py         # User CRUD, role management
│   │   ├── saves_handler.py         # Save slot grouping, sync state
│   │   ├── states_handler.py        # Save state CRUD
│   │   ├── screenshots_handler.py   # Screenshot CRUD
│   │   ├── firmware_handler.py      # BIOS/firmware CRUD
│   │   ├── collections_handler.py   # Regular/smart/virtual collections
│   │   ├── devices_handler.py       # Device registration, fingerprinting
│   │   ├── device_save_sync_handler.py # Cross-device save sync state
│   │   ├── client_tokens_handler.py # API token hash lookup
│   │   ├── play_sessions_handler.py # Play session ingest & aggregation
│   │   ├── sync_sessions_handler.py # Sync session lifecycle
│   │   └── stats_handler.py         # Library statistics
│   ├── filesystem/            # File I/O operations
│   │   ├── base_handler.py          # Path helpers, base class
│   │   ├── roms_handler.py          # ROM file reading, hashing
│   │   ├── assets_handler.py        # User assets storage
│   │   ├── firmware_handler.py      # BIOS file I/O
│   │   ├── platforms_handler.py     # Platform folder detection
│   │   └── resources_handler.py     # Artwork caching
│   └── metadata/              # Metadata provider handlers
│       ├── base_handler.py          # Base metadata handler
│       ├── igdb_handler.py          # IGDB provider
│       ├── moby_handler.py          # MobyGames provider
│       ├── ss_handler.py            # ScreenScraper
│       ├── sgdb_handler.py          # SteamGridDB
│       ├── ra_handler.py            # RetroAchievements
│       ├── hltb_handler.py          # HowLongToBeat
│       ├── hasheous_handler.py      # Hasheous hash-based lookup
│       ├── tgdb_handler.py          # TheGamesDB
│       ├── flashpoint_handler.py    # Flashpoint archive
│       ├── gamelist_handler.py      # gamelist.xml parser
│       ├── libretro_handler.py      # Libretro thumbnails DB lookup
│       ├── playmatch_handler.py     # PlayMatch algorithm
│       ├── launchbox_handler/       # LaunchBox (local + remote)
│       └── fixtures/                # Static metadata indexes
│           ├── mame_index.json          # MAME ROM → game info
│           ├── scummvm_index.json       # ScummVM identification
│           ├── ps1_serial_index.json    # PS1 serial → game
│           ├── ps2_serial_index.json    # PS2 serial → game
│           ├── ps2_opl_index.json       # PS2 OPL serials
│           └── psp_serial_index.json    # PSP serial → game
│
├── logger/                    # Logging setup
│   ├── logger.py              # Logger instance ("romm")
│   └── formatter.py           # Colored formatter, LOGGING_CONFIG
│
├── models/                    # SQLAlchemy ORM models
│   ├── base.py                # BaseModel (created_at, updated_at)
│   ├── user.py                # User, Role enum
│   ├── platform.py            # Platform
│   ├── rom.py                 # Rom, RomFile, RomMetadata, RomUser, RomNote
│   ├── collection.py          # Collection, SmartCollection, VirtualCollection
│   ├── assets.py              # Save, State, Screenshot
│   ├── device.py              # Device, SyncMode enum
│   ├── device_save_sync.py    # DeviceSaveSync
│   ├── firmware.py            # Firmware
│   ├── client_token.py        # ClientToken
│   ├── play_session.py        # PlaySession (per-user playtime tracking)
│   ├── sync_session.py        # SyncSession (device sync coordination)
│   └── fixtures/              # Seed data
│       └── known_bios_files.json    # Verified BIOS hashes
│
├── tasks/                     # Background job system
│   ├── tasks.py               # Base Task, PeriodicTask classes
│   ├── scheduled/             # Cron-scheduled tasks
│   │   ├── scan_library.py                    # Nightly library rescan
│   │   ├── sync_retroachievements_progress.py # Pull RA user progress
│   │   ├── update_switch_titledb.py           # Refresh Switch TitleDB
│   │   ├── update_launchbox_metadata.py       # Refresh LaunchBox data
│   │   ├── convert_images_to_webp.py          # Artwork WebP conversion
│   │   └── cleanup_netplay.py                 # Prune stale netplay rooms
│   └── manual/                # On-demand tasks
│       ├── cleanup_missing_roms.py       # Drop DB entries for missing files
│       ├── cleanup_orphaned_resources.py # Remove unreferenced artwork
│       └── sync_folder_scan.py           # Scan sync folder for new saves
│
├── utils/                     # Shared helpers
│   ├── __init__.py            # get_version()
│   ├── cache.py               # Redis fixture loading
│   ├── hashing.py             # CRC32, file hashing
│   ├── context.py             # Async context vars (aiohttp, httpx)
│   ├── database.py            # JSON/JSONB helpers, DB detection
│   ├── filesystem.py          # Path sanitization
│   ├── validation.py          # Input validation
│   ├── client_tokens.py       # Token generation
│   ├── datetime.py            # UTC helpers
│   ├── json_module.py         # Custom JSON encoder
│   ├── nginx.py               # X-Accel-Redirect responses
│   ├── router.py              # Custom APIRouter
│   ├── gamelist_exporter.py   # ES-DE gamelist.xml generation
│   ├── archive_7zip.py        # 7-Zip archive handling
│   ├── platforms.py           # Platform management
│   └── emoji.py               # Emoji utilities
│
├── tools/                     # Development utilities
│   └── xml_diagnostics.py     # XML diagnostic tool
│
├── tests/                     # Test suite
│   ├── conftest.py            # Pytest fixtures
│   └── ...                    # Mirrors backend structure
│
└── romm_test/                 # Test fixtures & data
    ├── assets/users/          # Test user saves
    ├── config/                # Test configuration
    ├── library/               # Test ROM library
    └── resources/roms/        # Test ROM resources

---

4. Application Lifecycle

Startup Sequence

1. alembic upgrade head          # Run database migrations
2. startup.main()                # Async startup tasks
   ├── Initialize scheduled jobs (RQ Scheduler)
   │   ├── cleanup_netplay
   │   ├── scan_library (if ENABLE_SCHEDULED_RESCAN)
   │   ├── update_switch_titledb
   │   ├── update_launchbox_metadata
   │   ├── convert_images_to_webp
   │   └── sync_retroachievements_progress
   └── Load fixture caches into Redis
       ├── mame_index.json
       ├── scummvm_index.json
       ├── ps1/ps2/psp serial indexes
       └── known_bios_files.json
3. uvicorn.run("main:app")       # Start ASGI server
   └── FastAPI lifespan
       ├── Create aiohttp.ClientSession
       ├── Create httpx.AsyncClient
       └── Store in app.state + context vars

Middleware Stack (execution order, outside-in)

Request →  CORS → CSRF → Authentication → Session (Redis) → Context Vars → Endpoint
Response ← CORS ← CSRF ← Authentication ← Session (Redis) ← Context Vars ← Endpoint

Layer	Middleware	Purpose
1	CORSMiddleware	Allow cross-origin requests (all origins)
2	CSRFMiddleware	Token-based CSRF protection (cookie + header)
3	AuthenticationMiddleware	HybridAuthBackend — Basic, Bearer, Session, OIDC
4	RedisSessionMiddleware	Cookie-based sessions stored in Redis
5	set_context_middleware	Inject aiohttp/httpx clients into context vars

Request Flow

HTTP Request
    │
    ├─ Middleware processes request (auth, session, CSRF)
    │
    ├─ FastAPI routes to endpoint handler
    │   └─ @protected_route checks scopes
    │
    ├─ Endpoint calls handler layer
    │   ├─ handler/database/*   → SQLAlchemy queries
    │   ├─ handler/metadata/*   → External API calls
    │   ├─ handler/filesystem/* → File I/O
    │   └─ handler/auth/*       → Token operations
    │
    ├─ Response schema (Pydantic) serializes output
    │
    └─ HTTP Response

---

5. Database Layer

Supported Databases

Database	Driver	Status
MariaDB 10.5+	mariadb+pymysql	Default
MySQL 8.0+	mysql+pymysql	Supported
PostgreSQL	postgresql+psycopg2	Supported

Engine & Session Setup

Location: handler/database/base_handler.py

sync_engine = create_engine(
    ConfigManager.get_db_engine(),
    pool_pre_ping=True,  # Connection health check
    echo=False,          # SQL logging (DEV_SQL_ECHO overrides)
)
sync_session = sessionmaker(bind=sync_engine, expire_on_commit=False)

Sessions are injected via the @begin_session decorator, which wraps handlers in a transaction context.

Base Model

Location: models/base.py

All models inherit BaseModel, providing:

Column	Type	Behavior
created_at	TIMESTAMP(tz=True)	Auto-set to UTC on creation
updated_at	TIMESTAMP(tz=True)	Auto-updated on modification

Constants: FILE_NAME_MAX_LENGTH=450, FILE_PATH_MAX_LENGTH=1000, FILE_EXTENSION_MAX_LENGTH=100

Entity-Relationship Diagram

                                    ┌──────────────┐
                          ┌────────>│  client_tokens│
                          │         └──────────────┘
                          │
                          │         ┌──────────────┐
                          ├────────>│   devices     │──────┐
                          │         └──────────────┘      │
                          │                                v
┌──────────┐              │         ┌──────────────┐    ┌──────────────────┐
│  users   │──────────────┼────────>│    saves      │<───│ device_save_sync │
└──────────┘              │         └──────────────┘    └──────────────────┘
     │                    │
     │                    │         ┌──────────────┐
     │                    ├────────>│   states      │
     │                    │         └──────────────┘
     │                    │
     │                    │         ┌──────────────┐
     │                    ├────────>│ screenshots   │
     │                    │         └──────────────┘
     │                    │
     │                    │         ┌──────────────┐
     │                    ├────────>│  rom_user     │
     │                    │         └──────────────┘
     │                    │              │
     │                    │              │
     │                    │         ┌──────────────┐      ┌──────────────┐
     │                    │    ┌───>│    roms       │<─────│  platforms   │
     │                    │    │    └──────────────┘      └──────────────┘
     │                    │    │         │                      │
     │                    │    │         ├────> rom_files       ├────> firmware
     │                    │    │         ├────> roms_metadata   │
     │                    │    │         ├────> rom_notes       │
     │                    │    │         ├────> saves           │
     │                    │    │         ├────> states          │
     │                    │    │         ├────> screenshots     │
     │                    │    │         └────> sibling_roms (self M:M)
     │                    │    │
     │         ┌──────────┴────┴──┐
     └────────>│   collections    │  (M:M via collections_roms)
               ├──────────────────┤
               │ smart_collections│  (filter-based, dynamic)
               ├──────────────────┤
               │virtual_collections│  (DB view, read-only)
               └──────────────────┘

Model Definitions

Users

Table: users

Column	Type	Notes
id	Integer	PK, autoincrement
username	String(255)	Unique, indexed
hashed_password	String(255)	Nullable (OIDC users)
email	String(255)	Unique, indexed, nullable
enabled	Boolean	Default True
role	Enum(VIEWER, EDITOR, ADMIN)	Default VIEWER
avatar_path	String(255)	Default ""
last_login	Timestamp	Nullable
last_active	Timestamp	Nullable
ra_username	String(255)	RetroAchievements username
ra_progression	JSON	RetroAchievements data
ui_settings	JSON	User preferences

Relationships: saves (1:M), states (1:M), screenshots (1:M), rom_users (1:M), notes (1:M), collections (1:M), smart_collections (1:M), devices (1:M, cascade), client_tokens (1:M, cascade)

---

Platforms

Table: platforms

Column	Type	Notes
id	Integer	PK
slug	String(100)	Indexed, canonical identifier
fs_slug	String(100)	Filesystem folder name
name	String(400)	Display name
custom_name	String(400)	User override
igdb_id, sgdb_id, moby_id, ss_id, ra_id, launchbox_id, hasheous_id, tgdb_id, flashpoint_id	Integer	External provider IDs
category	String(100)	Platform category
generation	Integer	Console generation
family_name / family_slug	String(1000)	Platform family
aspect_ratio	String(10)	Default "2 / 3"
missing_from_fs	Boolean	Default False

Computed properties: rom_count (subquery), fs_size_bytes (sum of ROM sizes)

Relationships: roms (1:M), firmware (1:M)

---

ROMs

Table: roms — the central entity

Column Group	Columns	Notes
Identity	id, platform_id (FK)	Core identifiers
External IDs	igdb_id, sgdb_id, moby_id, ss_id, ra_id, launchbox_id, hasheous_id, tgdb_id, flashpoint_id, hltb_id, gamelist_id	All indexed
Filesystem	fs_name, fs_name_no_tags, fs_name_no_ext, fs_extension, fs_path, fs_size_bytes	File info
Display	name, slug, summary	Game metadata
Provider metadata	igdb_metadata, moby_metadata, ss_metadata, ra_metadata, launchbox_metadata, hasheous_metadata, flashpoint_metadata, hltb_metadata, gamelist_metadata, manual_metadata	JSON blobs per provider
Media	path_cover_s, path_cover_l, url_cover, path_manual, url_manual, path_screenshots, url_screenshots	Cover art & screenshots
Classification	revision, version, regions, languages, tags	Game attributes
Hashes	crc_hash, md5_hash, sha1_hash, ra_hash	File integrity
State	missing_from_fs	Filesystem sync

Relationships: platform (M:1), files (1:M), saves (1:M), states (1:M), screenshots (1:M), rom_users (1:M), notes (1:M), metadatum (1:1), sibling_roms (M:M self-referential), collections (M:M)

---

ROM Files (table)

Table: rom_files

Tracks individual files within a ROM (archives can contain multiple files).

Column	Type	Notes
id	Integer	PK
rom_id	Integer	FK → roms
file_name, file_path	String	File identity
file_size_bytes	BigInteger	Size
crc_hash, md5_hash, sha1_hash, ra_hash	String(100)	Hashes
category	Enum	GAME, DLC, HACK, MANUAL, PATCH, UPDATE, MOD, DEMO, TRANSLATION, PROTOTYPE, CHEAT
missing_from_fs	Boolean	Sync state

---

ROM Metadata (Aggregated)

Table: roms_metadata — aggregated metadata from all providers

Column	Type
rom_id	Integer (PK, FK → roms)
genres	JSON
franchises	JSON
collections	JSON
companies	JSON
game_modes	JSON
age_ratings	JSON
player_count	String(100)
first_release_date	BigInteger (UNIX timestamp)
average_rating	Float

---

ROM User Data

Table: rom_user — per-user, per-ROM tracking

Column	Type	Notes
rom_id + user_id	FK composite	Unique constraint
is_main_sibling	Boolean	Primary version flag
last_played	Timestamp
backlogged	Boolean
now_playing	Boolean
hidden	Boolean
rating	Integer	0-5
difficulty	Integer
completion	Integer	Percentage
status	Enum	INCOMPLETE, FINISHED, COMPLETED_100, RETIRED, NEVER_PLAYING

---

ROM Notes

Table: rom_notes

Column	Type	Notes
id	Integer	PK
rom_id + user_id + title		Unique constraint
title	String(400)
content	Text
is_public	Boolean	Default False
tags	JSON

---

Collections

Table: collections — manually curated ROM lists

Column	Type	Notes
id	Integer	PK
user_id	FK → users
name	String(400)
description	Text
is_public	Boolean
is_favorite	Boolean
path_cover_s/l, url_cover	Text	Cover art

Linked to ROMs via collections_roms join table (M:M).

Table: smart_collections — dynamic, filter-based

Column	Type	Notes
filter_criteria	JSON	Query definition
rom_ids	JSON	Cached matching ROM IDs
rom_count	Integer	Cached count

View: virtual_collections — database view, read-only, excluded from migrations.

---

Assets (Saves, States, Screenshots)

All three share a similar structure:

Table	Extra Columns	Notes
saves	emulator, slot, content_hash	Device sync support
states	emulator	Save states
screenshots	—	In-game captures

Common columns: id, rom_id (FK), user_id (FK), file_name, file_path, file_size_bytes, missing_from_fs

Saves additionally link to device_save_sync for cross-device tracking.

---

Devices & Sync

Table: devices

Column	Type	Notes
id	String(255)	UUID, PK
user_id	FK → users
name, platform, client, client_version	String	Device info
sync_mode	Enum	API, FILE_TRANSFER, PUSH_PULL
sync_enabled	Boolean
last_seen	Timestamp

Table: device_save_sync — tracks per-device, per-save sync state

Column	Type
device_id + save_id	Composite PK
last_synced_at	Timestamp
is_untracked	Boolean

---

Client Tokens

Table: client_tokens — long-lived API tokens

Column	Type	Notes
id	Integer	PK
user_id	FK → users
name	String(255)	Display name
hashed_token	String(64)	SHA-256 hash, unique
scopes	String(1000)	Space-separated OAuth scopes
expires_at	Timestamp	Nullable
last_used_at	Timestamp

Token format: rmm_ + 64 hex chars (32-byte random)

---

Firmware

Table: firmware

Column	Type	Notes
id	Integer	PK
platform_id	FK → platforms
file_name, file_path	String
crc_hash, md5_hash, sha1_hash	String	Integrity
is_verified	Boolean	Matches known_bios_files.json

---

Alembic Migrations

80+ migration scripts in alembic/versions/. Key milestones:

Migration	Description
0009	Models refactor
0014, 0019	Asset filesystem refactoring
0020	Added created_at/updated_at to all tables
0021	ROM user associations
0022	Collection system
0023	Column nullability constraints
0024	Sibling ROM database views
0025	ROM hash tracking
0064	Performance indexes on updated_at
0068	Device + device_save_sync tables
0072	Client tokens table

Migrations support batch mode for SQLite and DB-specific SQL for MariaDB/MySQL/PostgreSQL.

---

6. API Endpoints

Base URL: /api Documentation: Swagger UI at /api/docs, ReDoc at /api/redoc Pagination: fastapi-pagination with LimitOffsetParams (limit, offset, total)

6.1 Authentication (/api)

Method	Path	Auth	Description
POST	/login	No	Session login (HTTP Basic)
POST	/logout	No	Logout (returns OIDC logout URL if configured)
POST	/token	No	OAuth2 token (password, refresh_token grants)
GET	/login/openid	No	OIDC login redirect
GET	/oauth/openid	No	OIDC callback
POST	/forgot-password	No	Request password reset
POST	/reset-password	No	Reset password with token

6.2 Users (/api/users)

Method	Path	Scope	Description
POST	/	ME_WRITE / USERS_WRITE	Create user (first user = admin)
POST	/invite-link	USERS_WRITE	Generate invite token
POST	/register	None	Register with invite token
GET	/	USERS_READ	List all users
GET	/identifiers	USERS_READ	Get user IDs
GET	/me	ME_READ	Current user profile
GET	/{id}	USERS_READ	Get user by ID
PUT	/{id}	ME_WRITE	Update user
DELETE	/{id}	USERS_WRITE	Delete user
POST	/{id}/ra/refresh	ME_WRITE	Refresh RetroAchievements data

6.3 Client Tokens (/api/client-tokens)

Method	Path	Scope	Description
POST	/	ME_WRITE	Create token
GET	/	ME_READ	List user's tokens
DELETE	/{id}	ME_WRITE	Delete token
PUT	/{id}/regenerate	ME_WRITE	Regenerate token
POST	/{id}/pair	ME_WRITE	Generate pair code
GET	/pair/{code}/status	None	Check pair status
POST	/exchange	None	Exchange pair code for token
GET	/all	USERS_READ	Admin: list all tokens
DELETE	/{id}/admin	USERS_WRITE	Admin: delete any token

6.4 Platforms (/api/platforms)

Method	Path	Scope	Description
POST	/	PLATFORMS_WRITE	Create platform
GET	/	PLATFORMS_READ	List platforms (with updated_after filter)
GET	/identifiers	PLATFORMS_READ	Get platform IDs
GET	/supported	PLATFORMS_READ	List supported platforms
GET	/{id}	PLATFORMS_READ	Get platform
PUT	/{id}	PLATFORMS_WRITE	Update platform
DELETE	/{id}	PLATFORMS_WRITE	Delete platform

6.5 ROMs (/api/roms)

Method	Path	Scope	Description
GET	/	ROMS_READ	List ROMs (paginated, filterable)
GET	/identifiers	ROMS_READ	Get ROM IDs
GET	/{id}	ROMS_READ	Get ROM details
PUT	/{id}	ROMS_WRITE	Update ROM metadata
PUT	/{id}/user	ME_WRITE	Update user-specific ROM data
DELETE	/{id}	ROMS_WRITE	Delete ROM
POST	/delete	ROMS_WRITE	Bulk delete
POST	/download/{id}/{file_name}	ROMS_READ	Download ROM
POST	/unidentified	ROMS_READ	Get unidentified ROMs

ROM Upload (Chunked)

Method	Path	Scope	Description
POST	/upload/init	ROMS_WRITE	Initialize upload session
POST	/upload/{id}/chunk	ROMS_WRITE	Upload chunk (max 64MB)
POST	/upload/{id}/complete	ROMS_WRITE	Finalize upload
GET	/upload/{id}/session	ROMS_WRITE	Check session status
DELETE	/upload/{id}	ROMS_WRITE	Cancel upload

ROM Files

Method	Path	Scope	Description
GET	/{id}/files	ROMS_READ	Get ROM file metadata
GET	/{id}/files/content/{name}	ROMS_READ	Download file (nginx X-Accel or direct)

6.6 Search (/api/search)

Method	Path	Scope	Description
GET	/roms	ROMS_READ	Search metadata across all providers
GET	/cover	ROMS_READ	Search SteamGridDB for cover art

6.7 Saves (/api/saves)

Method	Path	Scope	Description
POST	/	ASSETS_WRITE	Upload save (with optional device sync)
GET	/	ASSETS_READ	List saves (with device_id filter)
GET	/identifiers	ASSETS_READ	Get save IDs
GET	/summary	ASSETS_READ	Saves grouped by slot
GET	/{id}	ASSETS_READ	Get save
GET	/{id}/content	ASSETS_READ	Download save file
POST	/{id}/downloaded	DEVICES_WRITE	Confirm download (device sync)
PUT	/{id}	ASSETS_WRITE	Update save
POST	/delete	ASSETS_WRITE	Bulk delete
POST	/{id}/track	DEVICES_WRITE	Re-enable sync tracking
POST	/{id}/untrack	DEVICES_WRITE	Disable sync tracking

6.8 States (/api/states)

Method	Path	Scope	Description
POST	/	ASSETS_WRITE	Upload state
GET	/	ASSETS_READ	List states
GET	/identifiers	ASSETS_READ	Get state IDs
GET	/{id}	ASSETS_READ	Get state
PUT	/{id}	ASSETS_WRITE	Update state
POST	/delete	ASSETS_WRITE	Bulk delete

6.9 Screenshots (/api/screenshots)

Method	Path	Scope	Description
POST	/	ASSETS_WRITE	Upload screenshot
GET	/	ASSETS_READ	List screenshots
GET	/identifiers	ASSETS_READ	Get screenshot IDs
GET	/{id}	ASSETS_READ	Get screenshot
PUT	/{id}	ASSETS_WRITE	Update screenshot
POST	/delete	ASSETS_WRITE	Bulk delete

6.10 Devices (/api/devices)

Method	Path	Scope	Description
POST	/	DEVICES_WRITE	Register device (fingerprint dedup)
GET	/	DEVICES_READ	List devices
GET	/{id}	DEVICES_READ	Get device
PUT	/{id}	DEVICES_WRITE	Update device
DELETE	/{id}	DEVICES_WRITE	Delete device

6.11 Collections (/api/collections)

Method	Path	Scope	Description
POST	/	COLLECTIONS_WRITE	Create collection
GET	/	COLLECTIONS_READ	List collections
GET	/identifiers	COLLECTIONS_READ	Get collection IDs
GET	/{id}	COLLECTIONS_READ	Get collection
PUT	/{id}	COLLECTIONS_WRITE	Update collection
DELETE	/{id}	COLLECTIONS_WRITE	Delete collection
POST	/{id}/roms	COLLECTIONS_WRITE	Add ROM to collection
DELETE	/{id}/roms/{rom_id}	COLLECTIONS_WRITE	Remove ROM

6.12 Feeds (/api/feeds)

Method	Path	Description
GET	/webrcade	WebRcade feed format
GET	/tinfoil	Tinfoil custom index (Switch)
GET	/pkgi/ps3/{type}	PKGi PS3 database
GET	/pkgi/psvita/{type}	PKGi PS Vita database
GET	/pkgi/psp/{type}	PKGi PSP database
GET	/fpkgi/{platform}	FPKGi (PS4/PS5) format
GET	/kekatsu/{platform}	Kekatsu DS format
GET	/pkgj/psp/games	PKGj PSP games
GET	/pkgj/psp/dlc	PKGj PSP DLC
GET	/pkgj/psvita/games	PKGj PS Vita games
GET	/pkgj/psvita/dlc	PKGj PS Vita DLC
GET	/pkgj/psx/games	PKGj PSX games

6.13 Configuration (/api/config)

Method	Path	Scope	Description
GET	/	None	Get RomM configuration
POST	/system/platforms	PLATFORMS_WRITE	Add platform binding
DELETE	/system/platforms/{slug}	PLATFORMS_WRITE	Remove platform binding
POST	/system/versions	PLATFORMS_WRITE	Add version mapping
DELETE	/system/versions/{slug}	PLATFORMS_WRITE	Remove version mapping
POST	/system/exclusions	PLATFORMS_WRITE	Add exclusion pattern
DELETE	/system/exclusions	PLATFORMS_WRITE	Remove exclusion pattern

6.14 Tasks (/api/tasks)

Method	Path	Scope	Description
GET	/	TASKS_RUN	List all available tasks
GET	/status	TASKS_RUN	Status of all tasks
GET	/{id}	TASKS_RUN	Status of specific task
POST	/run/{name}	TASKS_RUN	Trigger task execution

6.15 Other Endpoints

Router	Path	Description
Heartbeat	GET /api/heartbeat	System info, version, metadata sources
Heartbeat	GET /api/heartbeat/metadata/{source}	Check metadata provider health
Heartbeat	GET /api/setup/library	Library structure info (wizard)
Heartbeat	POST /api/setup/platforms	Create platform folders (wizard)
Stats	GET /api/stats	Library statistics
Raw	HEAD /api/raw/assets/{path}	Check asset existence
Raw	GET /api/raw/assets/{path}	Serve raw asset file
Firmware	Standard CRUD	BIOS file management
Export	POST /api/export/gamelist-xml	Export ES-DE gamelist.xml
Export	POST /api/export/pegasus	Export Pegasus frontend metadata
Netplay	GET /api/netplay/list	List netplay rooms
Play Sessions	POST /api/play-sessions	Ingest play session from client
Play Sessions	GET /api/play-sessions	List play sessions (per user / ROM)
Sync	/api/sync/*	Device sync session coordination

The codebase exposes roughly 175 HTTP routes and 11 WebSocket handlers across 24 routers.

---

7. Authentication & Authorization

Authentication Methods

┌──────────────────────────────────────────────────────────────┐
│                    HybridAuthBackend                          │
│                                                              │
│  1. Check session cookie (romm_session)                      │
│     └─ Redis lookup → user from session["sub"]               │
│                                                              │
│  2. Check Authorization header                               │
│     ├─ "Basic ..." → bcrypt password verify                  │
│     ├─ "Bearer ..." → JWT validation (HS256)                 │
│     └─ "Bearer rmm_..." → Client API token (SHA-256 lookup)  │
│                                                              │
│  3. OIDC (if enabled)                                        │
│     └─ Token from OIDC provider → email match → user         │
│                                                              │
│  4. Kiosk mode (if enabled)                                  │
│     └─ Anonymous access with read-only scopes                │
│                                                              │
│  Falls through all methods → 401 Unauthorized                │
└──────────────────────────────────────────────────────────────┘

Token Types

Token	Format	Lifetime	Storage
Access Token	JWT (HS256)	30 min (configurable)	Client-side
Refresh Token	JWT with JTI	7 days (configurable)	JTI in Redis
Session	Cookie romm_session	14 days (configurable)	Redis
Client API Token	rmm_ + 64 hex chars	Configurable / never	SHA-256 hash in DB
CSRF Token	Signed cookie	Session lifetime	Cookie + header
Password Reset	JWT with JTI	10 minutes	JTI in Redis (one-time)
Invite Link	JWT with JTI	10 minutes	JTI in Redis (one-time)

Role-Based Access Control

Role	Scopes	Description
VIEWER	Read all + write own profile	Default role
EDITOR	VIEWER + write ROMs, platforms, assets	Content manager
ADMIN	EDITOR + user management, task execution	Full access

Scope Definitions

me.read / me.write           — Own profile
roms.read / roms.write       — ROM data
platforms.read / platforms.write — Platform data
assets.read / assets.write   — Saves, states, screenshots
devices.read / devices.write — Device management
firmware.read / firmware.write — BIOS files
collections.read / collections.write — Collections
users.read / users.write     — User management (admin)
tasks.run                    — Task execution

CSRF Protection

• Cookie: romm_csrftoken (signed with itsdangerous)
• Header: x-csrftoken
• Both must match and contain the authenticated user's ID
• Exempt: /api/token, /api/client-tokens/exchange, /api/client-tokens/pair/*/status, /ws, /netplay
• Skipped for requests with Authorization: Bearer or Authorization: Basic headers

Session Management

• Redis keys: session:{session_id}, user_sessions:{username}
• Cookie: romm_session (httponly, samesite=lax/strict)
• clear_user_sessions(user_id) on password change clears all sessions

---

8. Business Logic (Handlers)

8.1 Scan Handler (handler/scan_handler.py)

The core of RomM — orchestrates library scanning and metadata enrichment.

Scan Types:

Type	Behavior
NEW_PLATFORMS	Detect new platform folders only
QUICK	Scan new/unscanned ROMs
UPDATE	Rescan already-identified ROMs
UNMATCHED	Rescan ROMs without metadata
COMPLETE	Full rescan of everything
HASHES	Recalculate all file hashes

Scan Flow:

1. Detect platform folders in LIBRARY_BASE_PATH
2. For each platform:
   ├── Map filesystem slug to canonical platform (via config bindings)
   ├── Query metadata providers for platform info
   └── Create/update platform in DB
3. For each ROM file in platform:
   ├── Parse filename (extract name, tags, region, version)
   ├── Calculate file hashes (CRC32, MD5, SHA1)
   ├── Search metadata providers (in priority order):
   │   ├── IGDB
   │   ├── MobyGames
   │   ├── ScreenScraper
   │   ├── LaunchBox
   │   ├── RetroAchievements
   │   ├── Hasheous (hash-based matching)
   │   ├── Flashpoint
   │   ├── HLTB
   │   └── TheGamesDB
   ├── Download cover art and screenshots
   ├── Build aggregated metadata (RomMetadata)
   └── Create/update ROM in DB
4. Emit real-time progress via Socket.IO
5. Mark missing ROMs (files no longer on filesystem)

Search Term Normalization:

• Remove articles ("The", "A", "An")
• Strip punctuation and special characters
• Unicode normalization (NFKD)
• Jaro-Winkler similarity matching for fuzzy results

8.2 Database Handlers (handler/database/)

Each entity has a dedicated handler providing CRUD operations:

Handler	Key Operations
db_roms_handler	Advanced filtering (platform, genre, region, status), pagination, file association
db_platform_handler	Slug mapping, ROM count aggregation
db_users_handler	Role management, credential storage
db_saves_handler	Slot-based grouping, device sync tracking
db_collections_handler	ROM association, smart collection evaluation
db_stats_handler	Platform/ROM counts, storage usage, metadata coverage
db_devices_handler	Fingerprint deduplication
db_device_save_sync_handler	Cross-device sync state
db_client_tokens_handler	Hash-based lookup, scope management
db_play_sessions_handler	Play session ingestion and aggregation
db_sync_sessions_handler	Device sync session lifecycle (push/pull, SSH)

8.3 Metadata Handlers (handler/metadata/)

Each external provider has a handler that normalizes data into a common format:

Handler	Provider	Key Data
igdb_handler	IGDB	Game info, covers, screenshots, franchises
moby_handler	MobyGames	Publisher, genre classification
ss_handler	ScreenScraper	Regional metadata, box art, manuals
sgdb_handler	SteamGridDB	Grid artwork, logos, icons
ra_handler	RetroAchievements	Achievements, user progression
hltb_handler	HowLongToBeat	Playtime estimates
hasheous_handler	Hasheous	Hash-based ROM identification
tgdb_handler	TheGamesDB	Alternative metadata
flashpoint_handler	Flashpoint	Browser game archive
gamelist_handler	gamelist.xml	ES-DE format parser
libretro_handler	Libretro	Libretro thumbnails DB
playmatch_handler	PlayMatch	Game matching algorithm
launchbox_handler/	LaunchBox	Local + remote database, media

Priority system: Metadata sources are queried in configurable priority order. First match wins for each field, with manual overrides taking highest priority.

8.4 Filesystem Handlers (handler/filesystem/)

Handler	Responsibility
roms_handler	Read ROM files, calculate hashes, extract archives
assets_handler	Store/retrieve saves, states, screenshots
firmware_handler	BIOS file management, verification
platforms_handler	Platform folder creation and detection
resources_handler	Download and cache artwork

Supported archive formats: ZIP, 7Z, TAR, GZIP, BZ2, RAR

Special hash handling:

• CHD (Compressed Hunks of Data) v5: SHA1 extracted from header
• PICO-8 cartridges (.p8.png): special handling
• RetroAchievements hash (ra_hash): platform-specific algorithm via rahasher.py
• Non-hashable platforms (Switch, PS3/4/5): hashing skipped

8.5 Netplay Handler (handler/netplay_handler.py)

Manages real-time multiplayer rooms stored in Redis:

NetplayRoom:
    owner: str              # User ID
    players: dict           # sid → NetplayPlayerInfo
    peers: list[str]        # Peer IDs
    room_name: str
    game_id: str            # ROM ID
    domain: Optional[str]
    password: Optional[str]
    max_players: int

8.6 Play Sessions

Tracks per-user playtime events ingested from clients (web player, console mode, external launchers) via POST /api/play-sessions. Persisted in play_sessions table; aggregated into user profile stats and recent-activity feeds.

8.7 Device Sync Sessions

Coordinates save/state synchronization between devices using three sync modes (API, FILE_TRANSFER, PUSH_PULL). SyncSession tracks the lifecycle of a push/pull operation (including optional SSH-based file transfer — see SYNC_SSH_* env vars). Endpoints live in endpoints/sync.py; state is stored in the sync_sessions table.

8.8 Socket Handler (handler/socket_handler.py)

Manages two Socket.IO servers:

Server	Mount	Purpose
socket_handler	/ws	Scan progress, general notifications
netplay_socket_handler	/netplay	Netplay room management

Both use Redis as the message queue backend for horizontal scaling.

Scan Progress Events:

ScanStats:
    total_platforms, scanned_platforms, new_platforms
    total_roms, scanned_roms, new_roms, identified_roms
    scanned_firmware, new_firmware

---

9. External Integrations (Adapters)

Location: adapters/services/

Each adapter wraps an external API with authentication, retry logic, and type safety.

IGDB (Internet Game Database)

Property	Value
Auth	Twitch OAuth2 (client_id + client_secret → bearer token)
Data	Game metadata, covers, screenshots, age ratings, franchises, game modes
Rate limits	Retry logic with backoff
Config vars	IGDB_CLIENT_ID, IGDB_CLIENT_SECRET

MobyGames

Property	Value
Auth	API key
Data	Game metadata, publisher info, genre classification
Config var	MOBYGAMES_API_KEY

ScreenScraper

Property	Value
Auth	Device ID + user credentials
Data	Regional game metadata, box art, screenshots, manuals, bezels
Media types	Box 2D/3D, screenshot, video, manual, marquee, bezel
Config vars	SCREENSCRAPER_USER, SCREENSCRAPER_PASSWORD

SteamGridDB

Property	Value
Auth	Bearer token
Data	Grid artwork, logos, icons in multiple dimensions/styles
Filters	Style, dimension, MIME type
Config var	STEAMGRIDDB_API_KEY

RetroAchievements

Property	Value
Auth	API key (query parameter)
Data	Game achievements, user progression, award badges
Hash	Platform-specific hash via rahasher.py
Config var	RETROACHIEVEMENTS_API_KEY

Additional Providers

Provider	Handler	Description
LaunchBox	launchbox_handler/	Local XML database + remote API, platform mapping
HowLongToBeat	hltb_handler	Game playtime estimates
Hasheous	hasheous_handler	Hash-based ROM identification
TheGamesDB	tgdb_handler	Alternative game metadata
Flashpoint	flashpoint_handler	Browser game archive database
PlayMatch	playmatch_handler	Game matching algorithm

Static Fixture Data

Cached in Redis at startup from JSON files:

Fixture	Location	Purpose
mame_index.json	handler/metadata/fixtures/	MAME ROM name → game info
scummvm_index.json	handler/metadata/fixtures/	ScummVM game identification
ps1_serial_index.json	handler/metadata/fixtures/	PS1 serial code → game mapping
ps2_serial_index.json	handler/metadata/fixtures/	PS2 serial code → game mapping
ps2_opl_index.json	handler/metadata/fixtures/	PS2 OPL serial codes
psp_serial_index.json	handler/metadata/fixtures/	PSP serial code → game mapping
known_bios_files.json	models/fixtures/	Verified BIOS file hashes

---

10. Real-Time Communication (WebSockets)

Socket Architecture

Client  ←──Socket.IO──→  FastAPI (python-socketio)  ←──Redis PubSub──→  Workers

Scan Progress (/ws)

Events emitted to clients:

Event	Payload	When
scan:update_stats	ScanStats object	Each ROM/platform processed
scan:log	Log message	Scan log entries
scan:stop	—	Scan completed or cancelled

Netplay (/netplay)

Events:

Event	Direction	Description
open-room	Client → Server	Create netplay room
join-room	Client → Server	Join existing room
users-updated	Server → Client	Player list changed
Message relay	Bidirectional	Game data between peers

Redis-backed for horizontal scaling across multiple server instances.

---

11. Background Tasks & Scheduling

Job Queue System

Technology: RQ (Redis Queue)

Priority Queues:

Queue	Use Case
high_prio_queue	Urgent operations
default_queue	Standard background work
low_prio_queue	Long-running scans, cleanup

Scheduled Tasks

Configured via environment variables and managed by RQ Scheduler:

Task	Env Toggle	Default Cron	Description
scan_library	ENABLE_SCHEDULED_RESCAN	0 3 * * * (3 AM)	Full library rescan
update_switch_titledb	ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB	0 4 * * *	Update Switch game DB
update_launchbox_metadata	ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA	0 4 * * *	Refresh LaunchBox data
convert_images_to_webp	ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP	0 4 * * *	Image optimization
sync_retroachievements_progress	ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC	0 4 * * *	Sync RA user progress
cleanup_netplay	Always enabled	Periodic	Clean stale rooms

Manual Tasks

Triggered via POST /api/tasks/run/{task_name}:

Task	Description
cleanup_missing_roms	Remove DB entries for files no longer on disk
cleanup_orphaned_resources	Remove unused artwork/resource files
sync_folder_scan	Scan sync folder for new device saves

Filesystem Watcher

File: watcher.py

Uses watchfiles to monitor the library directory for changes. When enabled (ENABLE_RESCAN_ON_FILESYSTEM_CHANGE), triggers a rescan after a configurable delay (RESCAN_ON_FILESYSTEM_CHANGE_DELAY, default 5 minutes).

---

12. File System Management

Directory Layout

{ROMM_BASE_PATH}/                    # Default: /romm
├── library/                         # ROM files, organized by platform
│   ├── n64/
│   │   └── roms/
│   │       ├── Game1.z64
│   │       └── Game2.z64
│   ├── psx/
│   │   └── roms/
│   │       └── Game.bin
│   └── {platform_slug}/
│       ├── roms/                    # Configurable folder name
│       └── bios/                    # Firmware/BIOS files
│
├── resources/                       # Cached metadata assets
│   └── roms/
│       └── {rom_id}/
│           ├── cover_s.webp         # Small cover
│           ├── cover_l.webp         # Large cover
│           └── screenshots/
│
├── assets/                          # User-generated assets
│   └── users/
│       └── {user_id}/
│           └── {rom_id}/
│               ├── saves/
│               ├── states/
│               └── screenshots/
│
└── config/
    └── config.yml                   # YAML configuration

File Serving

Mode	Mechanism	When
Development	FileResponse (direct)	DEV_MODE=true
Production	Nginx X-Accel-Redirect	Default

Nginx receives an internal redirect header and efficiently serves the file from disk without passing bytes through the Python process.

Hash Calculation

Algorithm	Used For
CRC32	Quick integrity check, standard ROM identification
MD5	Content deduplication (saves), BIOS verification
SHA1	ROM identification, BIOS verification
RA Hash	RetroAchievements-specific hash (platform-dependent algorithm)

Hashing can be disabled per-installation via skip_hash_calculation in config.yml.

---

13. Caching (Redis)

Cache Architecture

Two Redis client instances:

Client	Type	Purpose
redis_client	Sync (with auto-decode)	Cache queries, session lookups
async_cache	Async	Async operations

Falls back to FakeRedis in test mode.

Cache Key Patterns

Pattern	TTL	Content
session:{id}	14 days	Session JSON
user_sessions:{username}	14 days	Set of session IDs
reset-jti:{jti}	10 min	Password reset token (one-time)
invite-jti:{jti}	10 min	Invite token (one-time)
refresh-jti:{jti}	7 days	Refresh token validation
romm:mame_index	Permanent	MAME game index
romm:scummvm_index	Permanent	ScummVM game index
romm:ps1_serials	Permanent	PS1 serial codes
romm:ps2_serials	Permanent	PS2 serial codes
romm:psp_serials	Permanent	PSP serial codes
romm:switch_titledb	Refreshed daily	Switch TitleDB
romm:known_bios	Permanent	Verified BIOS hashes
Upload sessions	24 hours	Chunked upload state
Netplay rooms	Dynamic	Active room state

---

14. Configuration

Environment Variables

Core

Variable	Default	Description
ROMM_BASE_PATH	/romm	Base data directory
ROMM_BASE_URL	http://0.0.0.0	Application base URL
ROMM_PORT	8080	Server port
DEV_MODE	false	Development mode
LOGLEVEL	INFO	Log level

Database

Variable	Default	Description
ROMM_DB_DRIVER	mariadb	mariadb, mysql, or postgresql
DB_HOST	—	Database host
DB_PORT	3306	Database port
DB_USER	—	Database user
DB_PASSWD	—	Database password
DB_NAME	romm	Database name

Redis

Variable	Default	Description
REDIS_HOST	127.0.0.1	Redis host
REDIS_PORT	6379	Redis port
REDIS_USERNAME	—	Redis username (ACL)
REDIS_PASSWORD	—	Redis password
REDIS_DB	0	Redis database number
REDIS_SSL	false	Enable SSL

Authentication

Variable	Default	Description
ROMM_AUTH_SECRET_KEY	—	Required. JWT/session signing key
OAUTH_ACCESS_TOKEN_EXPIRE_SECONDS	1800	30 minutes
OAUTH_REFRESH_TOKEN_EXPIRE_SECONDS	604800	7 days
SESSION_MAX_AGE_SECONDS	1209600	14 days
DISABLE_CSRF_PROTECTION	false	Disable CSRF
DISABLE_DOWNLOAD_ENDPOINT_AUTH	false	Allow unauthenticated downloads
DISABLE_USERPASS_LOGIN	false	Disable password login
KIOSK_MODE	false	Read-only anonymous access

OIDC

Variable	Default	Description
OIDC_ENABLED	false	Enable OpenID Connect
OIDC_PROVIDER	—	Provider URL
OIDC_CLIENT_ID	—	Client ID
OIDC_CLIENT_SECRET	—	Client secret
OIDC_REDIRECT_URI	—	Redirect URI
OIDC_USERNAME_ATTRIBUTE	preferred_username	Username claim
OIDC_CLAIM_ROLES	—	Roles claim name
OIDC_ROLE_VIEWER/EDITOR/ADMIN	—	Role mappings
OIDC_TLS_CACERTFILE	—	Custom CA bundle for OIDC calls
OIDC_RP_INITIATED_LOGOUT	false	Send logout to OIDC provider
OIDC_END_SESSION_ENDPOINT	—	End-session URL override

API Keys

Variable	Description
IGDB_CLIENT_ID + IGDB_CLIENT_SECRET	IGDB (via Twitch)
MOBYGAMES_API_KEY	MobyGames
SCREENSCRAPER_USER + SCREENSCRAPER_PASSWORD	ScreenScraper
STEAMGRIDDB_API_KEY	SteamGridDB
RETROACHIEVEMENTS_API_KEY	RetroAchievements

Feature Toggles

Variable	Default	Description
LAUNCHBOX_API_ENABLED	false	LaunchBox metadata
PLAYMATCH_API_ENABLED	false	PlayMatch matching
HASHEOUS_API_ENABLED	false	Hasheous identification
TGDB_API_ENABLED	false	TheGamesDB
FLASHPOINT_API_ENABLED	false	Flashpoint archive
HLTB_API_ENABLED	false	HowLongToBeat
DISABLE_EMULATOR_JS	false	Hide EmulatorJS player
DISABLE_RUFFLE_RS	false	Hide Ruffle Flash player

Task Scheduling

Variable	Default	Description
SCAN_TIMEOUT	14400	4-hour scan timeout
SCAN_WORKERS	1	Concurrent scan workers
TASK_TIMEOUT	—	RQ job timeout for manual tasks
TASK_RESULT_TTL	—	How long to keep job results
ENABLE_SCHEDULED_RESCAN	false	Auto library rescan
SCHEDULED_RESCAN_CRON	0 3 * * *	Rescan schedule
ENABLE_RESCAN_ON_FILESYSTEM_CHANGE	false	Watch for file changes
RESCAN_ON_FILESYSTEM_CHANGE_DELAY	5	Debounce delay (minutes)
SEVEN_ZIP_TIMEOUT	—	Timeout for 7-Zip extraction
REFRESH_RETROACHIEVEMENTS_CACHE_DAYS	—	RA cache TTL (days)

Device Sync

Variable	Default	Description
ENABLE_SYNC_FOLDER_WATCHER	false	Watch sync folder for new saves
SYNC_FOLDER_SCAN_DELAY	—	Debounce for sync folder scans
ENABLE_SYNC_PUSH_PULL	false	Enable scheduled push/pull sync
SYNC_PUSH_PULL_CRON	—	Cron schedule for push/pull
SYNC_SSH_KEYS_PATH	—	SSH keys path
SYNC_SSH_KNOWN_HOSTS_PATH	—	SSH known hosts path

YAML Configuration (config.yml)

exclude:
  platforms: ["arcade"]
  roms:
    single_file:
      extensions: [".txt", ".nfo"]
      names: ["readme"]
    multi_file:
      names: ["__MACOSX"]

filesystem:
  roms_folder: "roms" # Subfolder name for ROMs
  firmware_folder: "bios" # Subfolder name for BIOS
  skip_hash_calculation: false

system:
  platforms:
    snes: "snes" # fs_slug → canonical slug mappings
  versions:
    snes: "pal" # Platform version overrides

scan:
  priority:
    metadata: ["igdb", "moby", "ss"] # Provider priority
    artwork: ["sgdb", "igdb", "ss"]
    region: ["us", "eu", "jp"]
    language: ["en", "es", "ja"]
  media: ["box2d", "screenshot", "manual"]
  export_gamelist: false

emulatorjs:
  debug: false
  netplay:
    enabled: false
    ice_servers:
      - urls: "stun:stun.l.google.com:19302"
  settings:
    nes:
      option_name: option_value
  controls:
    nes:
      0: { 0: { value: "x" } }

Managed by ConfigManager (singleton pattern) which reads, validates, and writes the YAML file.

---

15. Error Handling

Exception Hierarchy

Exception
├── AuthCredentialsException          # 401 — Incorrect credentials
├── AuthenticationSchemeException      # 401 — Invalid auth scheme
├── UserDisabledException             # 401 — Account disabled
├── OAuthCredentialsException         # 401 — Invalid OAuth token
├── OIDCDisabledException             # 500 — OIDC not configured
├── OIDCNotConfiguredException        # 500 — OIDC feature disabled
│
├── PlatformNotFoundInDatabaseException    # 404
├── RomNotFoundInDatabaseException         # 404
├── CollectionNotFoundInDatabaseException  # 404
├── CollectionPermissionError              # 403
├── CollectionAlreadyExistsException       # 500
├── RomNotFoundInRetroAchievementsException # 404
├── SGDBInvalidAPIKeyException             # 401
│
├── FolderStructureNotMatchException  # Invalid library layout
├── PlatformNotFoundException         # Platform not found in FS
├── PlatformAlreadyExistsException    # Duplicate platform
├── RomsNotFoundException             # No ROMs for platform
├── RomAlreadyExistsException         # Duplicate ROM
├── FirmwareNotFoundException         # Firmware not found
│
├── ConfigNotWritableException        # Config file not writable
├── SchedulerException                # Task scheduling error
└── ScanStoppedException              # Scan cancelled

HTTP Status Codes

Code	Meaning
200	Success (GET, PUT)
201	Created (POST)
204	No Content (DELETE)
400	Bad Request
401	Unauthorized
403	Forbidden
404	Not Found
409	Conflict (duplicate)
500	Internal Server Error

---

16. Logging

Setup

Logger name: "romm" Level: Configurable via LOGLEVEL env var (default INFO)

Format

[LEVEL]: [RomM][module] [timestamp] message

Color Coding

Level	Color
DEBUG	Light Magenta
INFO	Green
WARNING	Yellow
ERROR	Light Red
CRITICAL	Red

Color behavior:

• FORCE_COLOR=true → Always use colors
• NO_COLOR=true → Strip ANSI codes
• Default → Colors enabled

Monitoring

• Sentry integration via SENTRY_DSN environment variable
• Release tagged as romm@{version}

---

17. Testing

Configuration

File: pytest.ini

• Async mode enabled
• Test database: romm_test
• Mock API keys pre-configured
• OIDC disabled for tests
• Log level: DEBUG

Test Structure

Tests mirror the backend directory structure under tests/:

tests/
├── conftest.py                    # Shared fixtures
├── adapters/services/             # API adapter tests
│   └── cassettes/                 # VCR recorded API responses
├── config/                        # Configuration tests
├── endpoints/                     # Endpoint integration tests
│   ├── test_auth.py
│   ├── test_collections.py
│   ├── roms/
│   └── sockets/
├── handler/                       # Handler unit tests
│   ├── auth/
│   ├── database/
│   ├── filesystem/
│   └── metadata/
├── logger/                        # Logger tests
├── models/                        # Model tests
├── tasks/                         # Task tests
└── utils/                         # Utility tests

Test Fixtures

Located in romm_test/:

• Test ROM library with real directory structure (n64, ps3, psp, psvita, psx)
• User asset fixtures
• Configuration fixtures
• VCR cassettes for external API responses (pre-recorded)

Key Patterns

• VCR cassettes for external API tests (reproducible without network)
• Test database (separate romm_test DB)
• FastAPI TestClient for endpoint integration tests
• Mock Redis via FakeRedis

---

Appendix: Key Design Patterns

Pattern	Where	Purpose
Three-tier architecture	Endpoints → Handlers → Models	Separation of concerns
Singleton	ConfigManager	Single config instance
Adapter pattern	adapters/services/	Normalize external APIs
Decorator pattern	@protected_route, @begin_session	Cross-cutting concerns
Context variables	utils/context.py	Request-scoped state (async-safe)
Repository pattern	handler/database/	Encapsulate data access
Observer pattern	Socket.IO events	Real-time updates
Priority queue	RQ with 3 priority levels	Task scheduling
Chunked upload	endpoints/roms/upload.py	Large file handling
X-Accel-Redirect	utils/nginx.py	Efficient file serving