OrientDB

The OrientDB connector provides Akka Stream sources and sinks for OrientDB.

For more information about OrientDB please visit the official documentation.

Artifacts

sbt
libraryDependencies += "com.lightbend.akka" %% "akka-stream-alpakka-orientdb" % "0.20"
Maven
<dependency>
  <groupId>com.lightbend.akka</groupId>
  <artifactId>akka-stream-alpakka-orientdb_2.12</artifactId>
  <version>0.20</version>
</dependency>
Gradle
dependencies {
  compile group: 'com.lightbend.akka', name: 'akka-stream-alpakka-orientdb_2.12', version: '0.20'
}

Usage

Sources, Flows and Sinks provided by this connector need dbUrl & credentials to access to OrientDB.

Scala
val url = "remote:127.0.0.1:2424/"
val dbName = "GratefulDeadConcertsScala"
val dbUrl = s"$url$dbName"
val username = "root"
val password = "root"
Full source at GitHub
Java
private static String url = "remote:127.0.0.1:2424/";
private static String dbName = "GratefulDeadConcertsJava";
private static String dbUrl = url + dbName;
private static String username = "root";
private static String password = "root";
Full source at GitHub

We will also need an ActorSystem and an ActorMaterializer.

Scala
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
Full source at GitHub
Java
system = ActorSystem.create();
materializer = ActorMaterializer.create(system);
Full source at GitHub

This is all preparation that we are going to need.

ODocument message

Now we can stream messages which contain OrientDB’s ODocument (in Scala or Java) from or to OrientDB by providing the ODatabaseDocumentTx to the OrientDBSource or the OrientDBSink.

Scala
val f2 = OrientDBSource(
  sink4,
  OrientDBSourceSettings(oDatabasePool = oDatabase)
).map { message =>
    message.oDocument.field[String]("book_title")
  }
  .runWith(Sink.seq)
Full source at GitHub
Java
CompletionStage<Done> f1 =
    OrientDBSource.create(source, OrientDBSourceSettings.create(oDatabase), null)
        .map(m -> OIncomingMessage.create(m.oDocument()))
        .runWith(
            OrientDBSink.create(sink1, OrientDBUpdateSettings.create(oDatabase)), materializer);
Full source at GitHub

Typed messages

Also, it’s possible to stream messages which contains any classes.

Java
public class source1 {

  private String book_title;

  public void setBook_title(String book_title) {
    this.book_title = book_title;
  }

  public String getBook_title() {
    return book_title;
  }
}

public class sink2 {

  private String book_title;

  public void setBook_title(String book_title) {
    this.book_title = book_title;
  }

  public String getBook_title() {
    return book_title;
  }
}
Full source at GitHub

Use OrientDBSource.typed and OrientDBSink.typed to create source and sink instead.

Java
CompletionStage<Done> f1 =
    OrientDBSource.typed(source, OrientDBSourceSettings.create(oDatabase), source1.class, null)
        .map(
            m -> {
              ODatabaseDocumentTx db = oDatabase.acquire();
              db.setDatabaseOwner(new OObjectDatabaseTx(db));
              ODatabaseRecordThreadLocal.instance().set(db);
              sink2 sink = new sink2();
              sink.setBook_title(m.oDocument().getBook_title());
              return OIncomingMessage.create(sink);
            })
        .runWith(
            OrientDBSink.typed(sink2, OrientDBUpdateSettings.create(oDatabase), sink2.class),
            materializer);
Full source at GitHub

Configuration

We can configure the source by OrientDBSourceSettings.

Scala (source)
final case class OrientDBSourceSettings(oDatabasePool: OPartitionedDatabasePool,
                                        maxPartitionSize: Int = Runtime.getRuntime.availableProcessors(),
                                        maxPoolSize: Int = -1,
                                        skip: Int = 0,
                                        limit: Int = 10)
Full source at GitHub
Parameter Default Description
maxPartitionSize OrientDBSource and OrientDBSink uses this for initializing DB Connections.
maxPoolSize -1 OrientDBSource and OrientDBSink uses this for initializing DB Connections.
skip OrientDBSource uses this property to fetch data from the DB.
limit OrientDBSource uses this property to fetch data from the DB.
dbUrl url to the OrientDB database.
username username to connect to OrientDB.
password password to connect to OrientDB.

Also, we can configure the sink by OrientDBUpdateSettings.

Scala (sink)
final case class OrientDBUpdateSettings(oDatabasePool: OPartitionedDatabasePool,
                                        maxPartitionSize: Int = Runtime.getRuntime.availableProcessors(),
                                        maxPoolSize: Int = -1,
                                        maxRetry: Int = 1,
                                        retryInterval: FiniteDuration = 5000 millis,
                                        bufferSize: Int = 10)
Full source at GitHub
Parameter Default Description
maxPartitionSize OrientDBSource and OrientDBSink uses this for initializing DB Connections.
maxPoolSize -1 OrientDBSource and OrientDBSink uses this for initializing DB Connections.
maxRetry 1 OrientDBSink uses this for retrying write operations to OrientDB.
retryInterval 5000 OrientDBSink uses this for retrying write operations to OrientDB.
bufferSize OrientDBSink uses this for retrieving data from DB.
dbUrl url to the OrientDB database.
username username to connect to OrientDB.
password password to connect to OrientDB.

Using OrientDB as a Flow

You can also build flow stages. The API is similar to creating Sinks.

Scala (flow)

val f1 = OrientDBSource( source, OrientDBSourceSettings(oDatabasePool = oDatabase) ).map { message: OOutgoingMessage[ODocument] => OIncomingMessage(message.oDocument) } .via( OrientDBFlow.create( sink5, OrientDBUpdateSettings(oDatabasePool = oDatabase) ) ) .runWith(Sink.seq)
Full source at GitHub
Java (flow)
CompletionStage<List<List<OIncomingMessage<ODocument, NotUsed>>>> f1 =
    OrientDBSource.create(source, OrientDBSourceSettings.create(oDatabase), null)
        .map(m -> OIncomingMessage.create(m.oDocument()))
        .via(OrientDBFlow.create(sink3, OrientDBUpdateSettings.create(oDatabase)))
        .runWith(Sink.seq(), materializer);
Full source at GitHub

Passing data through OrientDBFlow

When streaming documents from Kafka, you might want to commit to Kafka AFTER the document has been written to OrientDB.

Scala
// We're going to pretend we got messages from kafka.
// After we've written them to oRIENTdb, we want
// to commit the offset to Kafka

case class KafkaOffset(offset: Int)
case class KafkaMessage(book: Book, offset: KafkaOffset)

val messagesFromKafka = List(
  KafkaMessage(Book("Book 1"), KafkaOffset(0)),
  KafkaMessage(Book("Book 2"), KafkaOffset(1)),
  KafkaMessage(Book("Book 3"), KafkaOffset(2))
)

var committedOffsets = List[KafkaOffset]()

def commitToKakfa(offset: KafkaOffset): Unit =
  committedOffsets = committedOffsets :+ offset

val f1 = Source(messagesFromKafka)
  .map { kafkaMessage: KafkaMessage =>
    val book = kafkaMessage.book
    val id = book.title
    println("title: " + book.title)

    OIncomingMessage(new ODocument().field("book_title", id), kafkaMessage.offset)
  }
  .via(
    OrientDBFlow.createWithPassThrough(
      sink7,
      OrientDBUpdateSettings(oDatabase)
    )
  )
  .map { messages: Seq[OIncomingMessage[ODocument, KafkaOffset]] =>
    messages.foreach { message =>
      commitToKakfa(message.passThrough)
    }
  }
  .runWith(Sink.seq)

Await.ready(f1, Duration.Inf)
Full source at GitHub
Java
// We're going to pretend we got messages from kafka.
// After we've written them to OrientDB, we want
// to commit the offset to Kafka

List<Integer> committedOffsets = new ArrayList<>();
List<messagesFromKafka> messagesFromKafkas =
    Arrays.asList(
        new messagesFromKafka("Akka Concurrency", new KafkaOffset(0)),
        new messagesFromKafka("Akka in Action", new KafkaOffset(1)),
        new messagesFromKafka("Effective Akka", new KafkaOffset(2)));

Consumer<KafkaOffset> commitToKafka =
    new Consumer<KafkaOffset>() {
      @Override
      public void accept(KafkaOffset kafkaOffset) {
        committedOffsets.add(kafkaOffset.getOffset());
      }
    };

Source.from(messagesFromKafkas)
    .map(
        kafkaMessage -> {
          String book_title = kafkaMessage.getBook_title();
          return OIncomingMessage.create(
              new ODocument().field("book_title", book_title), kafkaMessage.kafkaOffset);
        })
    .via(OrientDBFlow.createWithPassThrough(sink6, OrientDBUpdateSettings.create(oDatabase)))
    .map(
        messages -> {
          ODatabaseDocumentTx db = oDatabase.acquire();
          db.setDatabaseOwner(new OObjectDatabaseTx(db));
          ODatabaseRecordThreadLocal.instance().set(db);
          messages
              .stream()
              .forEach(
                  message -> {
                    commitToKafka.accept(((KafkaOffset) message.passThrough()));
                  });
          return NotUsed.getInstance();
        })
    .runWith(Sink.seq(), materializer)
    .toCompletableFuture()
    .get(60, TimeUnit.SECONDS);
Full source at GitHub

Running the example code

The code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt.

> Test code requires OrientDB server running in the background. You can start one quickly using docker: > > docker run --rm -p 2424:2424 orientdb:latest

Scala
sbt
> orientdb/testOnly *.OrientDBSpec
Java
sbt
> orientdb/testOnly *.OrientDBTest
The source code for this page can be found here.