Serialization options for JavaScript services

You do not need to handle serialization for messages. Kalix functions serve gRPC interfaces, and the input and output messages are protobuf messages that get serialized to the protobuf format. However, for objects to be persisted such as Event Sourced events and snapshots you have a choice.

By default, Kalix will automatically detect if an emitted event or snapshot is a protobuf and handle them correctly, so we recommend using the default behavior. However, if you have specific requirements, you also can choose between primitive type and JSON (JavaScript Object Notation) serialization.

As a JavaScript developer, JSON is likely more familiar to you. When creating Event Sourced Entities, you can configure serialization for primitives or for JSON, using serializeAllowPrimitives or serializeFallbacktoJson, respectively.

Setting the serialization flag

The following example sets JSON serialization fo a snapshot:

JavaScript
const EventSourcedEntity = require("@kalix-io/kalix-javascript-sdk").EventSourcedEntity;

const entity = new EventSourcedEntity(
  ["shoppingcart.proto", "domain.proto"],
  "com.example.shoppingcart.ShoppingCart",
  "shopping-cart",
  {
    snapshotEvery: 5, // Usually you wouldn't snapshot this frequently, but this helps to demonstrate snapshotting
    includeDirs: ["./"],
    serializeFallbackToJson: true // Enables JSON support for persistence
  }
);
TypeScript
import { EventSourcedEntity } from "@kalix-io/kalix-javascript-sdk";

const entity: EventSourcedEntity = new EventSourcedEntity(
  ["shoppingcart.proto", "domain.proto"],
  "com.example.shoppingcart.ShoppingCart",
  "shopping-cart",
  {
    snapshotEvery: 5, // Usually you wouldn't snapshot this frequently, but this helps to demonstrate snapshotting
    includeDirs: ["./"],
    serializeFallbackToJson: true // Enables JSON support for persistence
  }
);

Receiving CloudEvents

Kalix uses the CloudEvents standard when receiving from and publishing to topics. The CloudEvents specification standardizes message metadata so that systems can integrate more easily.

Describing the structure of the message payload is the CloudEvents feature most important to Kalix.

An example of that is the capability to send serialized Protobuf messages and have Kalix deserialize them accordingly.

To allow proper reading of Protobuf messages from topics, the messages need to specify the message attributes:

  • Content-Type = application/protobuf

  • ce-specversion = 1.0

  • ce-type = fully qualified protobuf message name (e.g. shopping.cart.api.TopicOperation)

(The ce- prefixed attributes are part of the CloudEvents specification.)

The Protobuf rpc declaration uses the expected Protobuf message type and specifies the topic to subscribe to. You’ll normally want to share the exact Protobuf message declaration with the sending system.

syntax = "proto3";

import "google/protobuf/empty.proto";
import "google/protobuf/any.proto";
import "kalix/annotations.proto";

package shopping.cart.api;

message TopicOperation {
    string operation = 1;
}

service ShoppingCartTopicService {

    rpc ProtobufFromTopic(TopicOperation) returns (google.protobuf.Empty) {
        option (kalix.method).eventing.in = {
            topic:  "shopping-cart-protobuf-cloudevents"
        };
    }
}

Receiving JSON messages

Your Kalix service may subscribe to topics that use messages in JSON format. The messages must have the Content-Type attribute stating application/json.

The Protobuf rpc method receiving these JSON messages must be set up to receive Any.

syntax = "proto3";

import "google/protobuf/empty.proto";
import "google/protobuf/any.proto";
import "kalix/annotations.proto";

package shopping.cart.actions;

service ShoppingCartTopicService {
    rpc JsonFromTopic(google.protobuf.Any) returns (google.protobuf.Empty) {
        option (kalix.method).eventing.in = {
            topic:  "shopping-cart-json"
        };
    }
}

Publishing JSON messages to a topic

By default, when publishing a message to a topic, the protobuf message is serialized to bytes and published with the content-type/ce-datacontenttype application/protobuf, and will also contain the metadata entry ce-type specifying the specific protobuf message type.

This is convenient when the consumer is another Kalix service that can handle protobuf messages.

In many cases the consumer may be an external service though, and in such a case another serialization format for the messages can make sense. For such a use case the Kalix JavaScript SDK supports emitting JSON messages.

To publish JSON messages to a topic, mark the return type of the message as google.protobuf.Any. The object returned with replies.message() will be serialized to string format and published to the topic with content type Content-Type attribute stating application/json.

import "google/protobuf/any.proto";
import "google/protobuf/empty.proto";
import "kalix/annotations.proto";

message KeyValue {
  string key = 1;
  int32 value = 2;
}

service MyService {
  option (kalix.codegen) = {
    action: {}
  };

  rpc Consume(google.protobuf.Any) returns (google.protobuf.Empty) {
    option (kalix.method).eventing.in = {
      topic:  "notifications"
    };
  }

  rpc Produce(KeyValue) returns (google.protobuf.Any) { (1)
    option (kalix.method).eventing.out = { (2)
      topic:  "notifications" (3)
    };
  }

}
1 return google.protobuf.Any from the method
2 annotate the Protobuf rpc method with (kalix.method).eventing
3 use out and topic to publish to a topic

In the service implementation, return an arbitrary object, it will be serialized to JSON for the topic:

async Produce(request) {
  return replies.message({ (2)
    arbitrary: "json"
  })
}