Tutorial for synchronized APIs in the Map Editor

This tutorial explains the synchronization mechanism of the collaborative Map Editor, focusing on how user actions are communicated between the frontend and the backend. It covers what common CRUD operations for different map elements, the structure of action requests, and the benefits of batch operations. At the end we look the implementation of PATCH /api/maps/{map_id}/layers/plants/plantings as an example.

Map Editor Synchronization

When a user makes a modification to a map, other users see that change reflected in real-time without needing to refresh the webpage. All operations are synchronized through a system of update events, referred to as Actions in the backend and RemoteActions in the frontend. See backend/src/model/dto/actions.rs and frontend/src/features/map_planning/store/RemoteActions.ts. Since the tutorial is written from the perspective of the backend, they are referred to as actions here. When opening the Map Editor, the frontend establishes a connection to the Server-Sent Events (SSE) stream at /api/updates/maps?map_id=<map_id>&user_id=<user_id>.

Common Interface

All APIs that act on maps support standard CRUD (Create, Read, Update, Delete) operations:

Get: Retrieve elements.
Post: Create new elements.
Patch: Update existing elements. Note: different types of updates can be processed through the same endpoint.
Delete: Remove elements.

Please use appropriate HTTP methods and status codes (See our API guidelines).

Note that the Delete operation implements a soft delete for layers and maps, allowing for recovery.

If the request changes the state of the map (Post, Put, Patch, Delete) the request body is wrapped in ActionDtoWrapper (backend/src/model/dto/core.rs). ActionDtoWrapper adds an actionId to the request. This actionId is generated and supplied by the frontend to recognize its own actions within the SSE stream of events. The backend updates the database and then sends out an update to all users on the same map.

E.g., the API to place a new Planting onto the map via POST /api/maps/<map_id>/layers/plants/plantings sends the following request body.

{
    "actionId": "<uuid>",
    "dto": { ... }
}

The backend processes the request and sends out the CreatePlanting action. This is the JSON payload received by the SSE stream of all listeners.

{
  "action": {
    "payload": [
      {
        "id": "<planting_uuid>",
        "sizeX": 300,
        "sizeY": 300,
        "x": -315,
        "y": -959,
        ... more planting attributes
      }
    ],
    "type": "CreatePlanting"
  },
  "actionId": "<uuid>",
  "userId": "<uuid>"
}

All APIs that operate on a map should be under /api/maps/<map_id>/... (See our API guidelines). To register a new API we need to create a file in the backend/src/controller directory and register the routes in backend/src/config/routes.rs.

Common Properties

The id of each map element (layers, plantings, drawings, areas...) is a UUID that is generated by the frontend. The frontend optimistically displays the change in the UI and then updates fields from the response body when the request succeeds. In case the update failed (HTTP status code >= 300) it undoes the changes visually.

Metadata is stored for some elements. At the time of writing, the tables maps and plantings have metadata. If you want to add metadata to a table, please use these columns:

created_by: User UUID, set on creation
created_at: Timestamp of the creation, only set once
modified_by: User UUID, set on each update
modified_at: Timestamp of the last update, set on each update

Batch operations

Many operations on the map use a list of items instead of a single item. For instance, to move one or more plantings on the map we use the request PATCH /api/maps/{map_id}/layers/plants/plantings with a list of plantings.

{
  "actionId": "<action_id>",
  "dto": {
    "type": "move",
    "content": [
      {
        "id": "<planting_uuid_1>",
        "x": 10,
        "y": 10
      },
      {
        "id": "<planting_uuid_2>",
        "x": 20,
        "y": 20
      }
    ]
  }
}

This is useful because the user can move many plantings at once by selecting multiple plantings and dragging them.

Add new layer type to all maps

To add a new layer type to all maps, we need to follow two steps:

Migrate the layers table to add the layer to all existing maps. (You can find examples in the migrations; just look for "INSERT INTO layers").
The create() function in the map service in backend/src/service/map.rs inserts all layers into a newly created map. Update this function to also include the new layer type upon creation.

Example

Let's look at the HTTP Patch for plantings, that includes the MovePlanting update. This is the update function in backend/src/controller/plantings.rs at the time of writing.

#![allow(unused)]
fn main() {
#[utoipa::path(
    context_path = "/api/maps/{map_id}/layers/plants/plantings",
    params(
        ("map_id" = i32, Path, description = "The id of the map the layer is on"),
    ),
    request_body = ActionDtoWrapperUpdatePlantings,
    responses(
        (status = 200, description = "Update plantings", body = Vec<PlantingDto>)
    ),
    security(
        ("oauth2" = [])
    )
)]
#[patch("")]
pub async fn update(
    path: Path<i32>,
    update_planting: Json<ActionDtoWrapper<UpdatePlantingDto>>,
    user_info: UserInfo,
    pool: SharedPool,
    broadcaster: SharedBroadcaster,
) -> Result<HttpResponse> {
    let map_id = path.into_inner();

    let ActionDtoWrapper { action_id, dto } = update_planting.into_inner();

    let updated_plantings = plantings::update(dto.clone(), map_id, user_info.id, &pool).await?;

    let action = match &dto {
        UpdatePlantingDto::Transform(dto) => {
            Action::new_transform_planting_action(dto, user_info.id, action_id)
        }
        UpdatePlantingDto::Move(dto) => {
            Action::new_move_planting_action(dto, user_info.id, action_id)
        }
        // ... Other update types here
    };

    broadcaster.broadcast(map_id, action).await;

    Ok(HttpResponse::Ok().json(updated_plantings))
}
}

Let's start with the two attributes (#[utoipa::path(...)], #[patch("")]).

utoipa is used to generate the OpenAPI documentation, it doesn't change the functionality of the API. The file backend/src/config/api_doc.rs must be edited in combination with the information in the attribute.

The attribute #[patch("")] is from the actix_web crate to denote what HTTP method is used. The path must additionally be registered as a route in backend/src/config/routes.rs.

Function parameters can use the wrappers:

Path: to capture variables in the URL's path segment.
Query: to capture variables in the URL's query parameters (not used in this example).
Json: to use the request body. The request body is already validated and parsed by the framework and converted to the Rust type ActionDtoWrapper<UpdatePlantingDto>.

Apart from these parameters, requests can also inject globally available data wrappers. Here we use:

UserInfo: to get information about the user who sent the request.
SharedPool: to gain access to the database.
SharedBroadcaster: to use the SSE Broadcaster.

As we can see, the request body is wrapped in ActionDtoWrapper since the request causes an action to be sent. Typically a request calls out to one or more services, which validate the input and update the database. After we await the call to plantings::update(), we need to send the appropriate action to the user. The match statement creates the appropriate action payload depending on the type of update. We tell the broadcaster to send off the action to all listeners before returning back 200 OK.

PermaplanT