OpenTelemetry tries to send traces to a default `localhost:4317`
endpoint if no configuration is provided, which is not what we want if
users don't configure OpenTelemetry explicitly.
This change sets the `OTEL_SDK_DISABLED` environment variable to
`true` if no `OTEL_` prefixed environment variables are found, which
disables the OpenTelemetry SDK.
Guarantee that cache is initialized during startup, and only once,
instead of every time a `MetadataHandler` object is instantiated.
Also, improve logic to determine `fixtures` paths.
For steps that need to run before the web application starts, such as
scheduling tasks, this new `startup.py` script is introduced.
This fixes a recently introduced issue where task scheduling was not
being triggered, because of it being included in the
`if __name__ == "__main__":` block, which is not executed when
the application is run by Gunicorn in production environments.
We do not include this logic as part of FastAPI's lifespan
implementation, as running multiple workers with Gunicorn would
cause this logic to be executed multiple times.
This change introduces OpenTelemetry dependencies, and uses
`opentelemetry-instrument` auto-instrumentation to allow users to
configure OpenTelemetry settings via environment variables [1].
The only custom environment variable added is
`OTEL_SERVICE_NAME_PREFIX`, which allows users to set a prefix to the
service names included by RomM to differentiate between `api`, `worker`,
and `watcher` services.
The instrumentation of RQ workers and file watcher will be added in
subsequent pull requests.
[1] https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
* Added `linux-headers` back, but only for development stage.
* Fixed initialization script, as `uv` is not included in the final
Docker image.
* Initialize variable `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA`.
* Rename file to `Dockerfile.dockerignore`. As the `Dockerfile` is not
in the root directory, the Docker spec requires the file to be
named `<dockerfile_name>.dockerignore`.
* Update ignore rules, and include `.mypy_cache/` directory.
The `watchfiles` library supports event batching, which allows us to
process multiple filesystem changes in a single run.
This change also avoids database calls in the watcher as much as
possible.
This change replaces our custom `scheduler.py` script with the
`rqscheduler` command, allowing us to run the RQ scheduler as a
separate, low-memory process, by avoiding the need to maintain
the Python app in memory.
* Remove `scheduler.py` script.
* Move initialization of scheduled tasks to `worker.py`.
* Update `docker/init_scripts/init` to start the `rqscheduler`
command instead of the custom script.
* Fix scheduled tasks' `func` paths to the new project structure.
* Temporarily use a fork of `rq-scheduler` to support
username and SSL settings in the `rqscheduler` command.
The `production-stage` stage was depending on the `python3` package from
Alpine, which at the moment of writing is still Python 3.12.
To avoid relying on Alpine's package and releases, we now copy the
Python installation directly from its official Docker image.
Other options were tested but did not work:
- Trying to install Python 3.13 using `apk`. As mentioned, Alpine does
not support Python 3.13 yet.
- Installing Python 3.13 using `pyenv`. The installation worked, but the
generated image was too large. Related `pyenv` discussion:
https://github.com/orgs/pyenv/discussions/2868
- Installing Python 3.13 using `uv`. Only worked for `amd64`
architecture, but the installation failed on `arm64`.
The previous attempt to upgrade RALibretro before a new tagged version
was available (#1970) needed to be reverted, as the RAHasher tool failed
with segmentation faults in some scenarios.
Bump `RALibretro` to the latest commit from the upstream repository.
Removes the need to use an older version of Alpine Linux, but requires
some changes to the `RALibretro` source code to compile successfully.
The `nginx` image is still not available for Alpine 3.22, but we can
also upgrade to its latest patch version (`1.27.5`).
