# Description of Changes
Followup to #3802; was causing issues in modules in private. Rust 2024
now requires `unsafe()` around certain attributes, but the `settings`
macro was added after I first opened that PR, so I didn't wrap it.
Additionally, `settings` wasn't tested in this repo, so it wasn't
caught.
# Expected complexity level and risk
1
# Testing
- [x] Added use of `#[spacetimedb::settings]` in `module-test`
## Description of Changes
This PR primarily affects the `bindings-macro` and `schema` crates to
review:
### Core changes
1. Replaces the `name` macro with `accessor` for **Tables, Views,
Procedures, and Reducers** in Rust modules.
2. Extends `RawModuleDefV10` with a new section for:
* case conversion policies
* explicit names
New sections are not validated in this PR so not functional.
3. Updates index behavior:
* Index names are now always **system-generated** for clients. Which
will be fixed in follow-up PR when we start validating RawModuleDef with
explicit names.
* The `accessor` name for an index is used only inside the module.
## Breaking changes (API/ABI)
1. **Rust modules**
* The `name` macro must be replaced with `accessor`.
2. **Client bindings (all languages)**
* Index names are now system-generated instead of using explicitly
provided names.
**Complexity:** 3
A follow-up PR will reintroduce explicit names with support for case
conversion.
---------
Co-authored-by: rekhoff <r.ekhoff@clockworklabs.io>
Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com>
Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com>
# Description of Changes
Makes the `sender` field on `ViewContext` private and exposes a
`sender()` method. Does the same for `ReducerContext` and
`ProcedureContext`.
The purpose of this change: So that the host can determine if/when a
view invokes or reads the `sender`.
Currently, because `sender` is a field, the host assumes that it is
always read. This means views must be materialized per client, even if
the view doesn't actually depend on `sender`, resulting in data
duplication.
The initial solution for this problem was `AnonymousViewContext` which
doesn't have a `sender` field. The better solution is to make `sender` a
method so that it can call into the host and record when it's actually
invoked.
Note, this patch only updates the module API, so the current
implementation does not change. `ViewContext` views are still duplicated
across clients. Changing this requires a new host syscall and for
`sender()` to invoke that syscall. This however is backwards compatible
and can be done anytime after the module APIs for the other languages
(C#, TypeScript, C++) are updated.
Also note that `ReducerContext` and `ProcedureContext` were updated
purely for consistency. There are currently no plans to track reads of
`sender` in these contexts.
# API and ABI breaking changes
Breaks the rust module api.
# Expected complexity level and risk
1
# Testing
N/A
# Description of Changes
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3729
I genuinely don't know what came over me.
# API and ABI breaking changes
None
# Expected complexity level and risk
1.5 very straightforward but not strictly trivial
# Testing
Adds automated integration tests (written in Rust and run with `cargo
test`, although note this comment from @matklad about integration tests
for the future
https://internals.rust-lang.org/t/running-test-crates-in-parallel/15639/2):
- [x] Can publish an updated module if no migration is required
- [x] Can publish an updated module if auto-migration is required (with
the yes-break flag true/false)
- [x] Cannot publish if a manual migration is required
- [x] Can publish if a manual migration is required but the user
specified `--delete-data`
- [x] Can publish if a manual migration is required by the user
specified `--delete-data=on-conflict`
- [x] No data deletion occurs if no migration is required and
`--delete-data=on-conflict` is specified
---------
Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com>
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
Co-authored-by: Phoebe Goldman <phoebe@clockworklabs.io>
Co-authored-by: John Detter <4099508+jdetter@users.noreply.github.com>
# Description of Changes
Closes#3517 .
With this PR, procedures (at least, those defined in Rust modules) can
perform HTTP requests! This is performed through a new field on the
`ProcedureContext`, `http: HttpClient`, which has a method `send` for
sending an `http::Request`, as well as a convenience wrapper `get`.
Internally, these methods hit the `procedure_http_request` ABI call /
host function, which uses reqwest to perform an HTTP request. The
request is run with a user-configurable timeout which defaults and is
clamped to 500 ms.
Rather than exposing the HTTP stream to modules, we download the entire
response body immediately, within the same timeout.
I've added an example usage of `get` to `module-test` which performs a
request against `localhost:3000` to read its own schema/moduledef.
This PR also makes all procedure-related definitions in the Rust module
bindings library `#[cfg(feature = "unstable")]`, as per #3644 . The
rename of the `/v1/database/:name/procedure/:name` route is not included
in this PR, so this does not close#3644 .
Left as TODOs are:
- Metrics for recording request and response size.
- Improving performance by stashing a long-lived `reqwest::Client`
someplace.
Currently we build a new `Client` for each request.
- Improving performance (possibly) by passing the request-future to the
global tokio executor
rather than running it on the single-threaded database executor.
# API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is
not unstable in any meaningful way (we can't really do that). Marks
unreleased APIs as unstable. Does not affect any pre-existing
already-released APIs or ABIs.
# Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which
prevents these connections from being truly long-lived, they're still
potentially long-lived on the scale of Tokio futures. It's possible that
running them on the database core is problematic in some way, and so
what I've left as a performance TODO could actually be a
concurrency-correctness issue.
# Testing
- [x] Manually wrote and executed some procedures which make HTTP
requests.
- [x] Added two automated tests to the `sdk-test` suite,
`procedure::http_ok` and `procedure::http_err`, which make successful
and failing requests respectively, then return its result. A client then
makes some assertions about the result.
---------
Co-authored-by: Noa <coolreader18@gmail.com>
# Description of Changes
Adds `ProcedureContext::{with_tx, try_with_tx}`.
Fixes https://github.com/clockworklabs/SpacetimeDB/issues/3515.
# API and ABI breaking changes
None
# Expected complexity level and risk
2
# Testing
An integration test `test_calling_with_tx` is added.
# Description of Changes
Specifically generate client table handles for views.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
Added a `#[view]` to `module-test` and generated new client snapshots
# Description of Changes
This commit adds a macro attribute `#[procedure]` which applies to
functions, and various in-WASM machinery for defining, calling and
running procedures.
A simple example of a procedure, included in `modules/module-test`:
```rust
fn sleep_one_second(ctx: &mut ProcedureContext) {
let prev_time = ctx.timestamp;
let target = prev_time + Duration::from_secs(1);
ctx.sleep_until(target);
let new_time = ctx.timestamp;
let actual_delta = new_time.duration_since(prev_time).unwrap();
log::info!("Slept from {prev_time} to {new_time}, a total of {actual_delta:?}");
}
```
We intend eventually to make procedures be `async` functions (with the
trivial `now_or_never` executor from `future-util`), but I found that
making the types work for this was giving me a lot of trouble, and
decided to put it off in the interest of unblocking more parallelizable
work.
Host-side infrastructure for executing procedures is not included in
this commit. I have a prototype working, but cleaning it up for review
and merge will come a bit later.
One item of complexity in this PR is enabling scheduled tables to
specify either reducers or procedures, while still providing
compile-time diagnostics for ill-typed scheduled functions (as opposed
to publish-time). I had to rewrite the previous
`schedule_reducer_typecheck` into a more complex `schedule_typecheck`
with a trait `ExportFunctionForScheduledTable`, which takes a "tacit
trait parameter" encoding reducer-ness or procedure-ness, as described
in https://willcrichton.net/notes/defeating-coherence-rust/ .
The trait name `ExportFunctionForScheduledTable` is user-facing in the
sense that it will appear in compiler diagnostics in ill-typed modules.
As such, I am open to bikeshedding.
# API and ABI breaking changes
Adds a new user-facing API, which we intend to change before releasing.
# Expected complexity level and risk
2? Mostly pretty mechanical changes to macros and bindings.
# Testing
- [x] Added a procedure definition to `module-test`, saw that it
typechecks.
- [x] Executed same procedure definition using #3390 , the prototype
implementation this PR draws from.
# Description of Changes
PR introduces support for column-level default values via a new
`#[default(...)]` attribute.
It also validates, `default` macro is not used along with `primary_key`,
`unique` or `auto_inc`.
# API and ABI breaking changes
NA
# Expected complexity level and risk
2
# Testing
Start using macro in `module-test`.
---------
Co-authored-by: James Gilles <jameshgilles@gmail.com>
# Description of Changes
Apparently, I missed several license files in #3002. I'm not sure what
method I was using to find them, but apparently it was insufficient.
**This replaces all empty `LICENSE` files with an explicit (symlink to)
BSL license, and all apache licenses with symlinks to the root apache
license.** This PR does not intentionally change any license terms, so
if you see one that changed, **it's a mistake**.
# API and ABI breaking changes
None
# Expected complexity level and risk
1
# Testing
```bash
$ find . -name '*LICENSE*' -type f | grep -v '\.meta$'
./crates/sqltest/standards/LICENSE # this one is an external library that we are not allowed to re-license
./LICENSE.txt # this is the root license
```
---------
Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
