Building microservices with Scala, functional domain models and Spring Boot

@crichardson Building microservices with Scala, functional domain models and Spring Boot Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com

@crichardson Presentation goal Share my experiences with building an application using Scala, functional domain models, microservices, event sourcing, CQRS, and Spring Boot

@crichardson About Chris

@crichardson About Chris Founder of a buzzword compliant (stealthy, social, mobile, big data, machine learning, ...) startup Consultant helping organizations improve how they architect and deploy applications using cloud, micro services, polyglot applications, NoSQL, ...

@crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices

@crichardson Let’s imagine that you are building a banking app...

@crichardson Domain model Account balance open(initialBalance) debit(amount) credit(amount) TransferTransaction fromAccountId toAccountId amount

@crichardson Tomcat Traditional application architecture Browser/ Client WAR/EAR RDBMS Customers Accounts Transactions BankingBanking UI develop test deploy Simple to Load balancer scale Spring MVC Spring Hibernate ... HTML REST/JSON

@crichardson Problem #1: monolithic architecture Intimidates developers Obstacle to frequent deployments Overloads your IDE and container Obstacle to scaling development Requires long-term commitment to a technology stack

@crichardson Solution #1: use a microservice architecture Banking UI Account Management Service Transaction Management Service Account Database Transaction Database Standalone services

@crichardson Problem #2: relational databases Scalability Distribution Schema updates O/R impedance mismatch Handling semi-structured data

@crichardson Solution #2: use NoSQL databases Avoids the limitations of RDBMS For example, text search Solr/Cloud Search social (graph) data Neo4J highly distributed/available database Cassandra ...

@crichardson But now we have problems with data consistency!

@crichardson Problem #3: Microservices = distributed data management Each microservice has it’s own database Some data is replicated and must be kept in sync Business transactions must update data owned by multiple services Tricky to implement reliably without 2PC

@crichardson Problem #4: NoSQL = ACID-free, denormalized databases Limited transactions, i.e. no ACID transactions Tricky to implement business transactions that update multiple rows, e.g. http://bit.ly/mongo2pc Limited querying capabilities Requires denormalized/materialized views that must be synchronized Multiple datastores (e.g. DynamoDB + Cloud Search ) that need to be kept in sync

@crichardson Solution to #3/#4: Event-based architecture to the rescue Microservices publish events when state changes Microservices subscribe to events Synchronize replicated data Maintains eventual consistency across multiple aggregates (in multiple datastores)

@crichardson Eventually consistent money transfer Message Bus Transaction management service Account management service transferMoney() Publishes: Subscribes to: Subscribes to: publishes: TransferTransactionCreatedEvent AccountDebitedEvent DebitRecordedEvent AccountCreditedEvent TransferTransactionCreatedEvent DebitRecordedEvent AccountDebitedEvent AccountCreditedEvent

@crichardson But reliably generating events is difﬁcult Must atomically update datastore and publish event(s) Non-option: datastore and message broker use 2PC Use datastore as message queue Local transaction updates state and publishes message See BASE: An Acid Alternative, http://bit.ly/ebaybase But Business logic and event publishing code intertwined Tricky to implement with aggregate-oriented NoSQL database

@crichardson Event sourcing For each aggregate: Identify (state-changing) domain events Deﬁne Event classes For example, Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEvent ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent

@crichardson Persists events NOT current state Account balance open(initial) debit(amount) credit(amount) AccountOpened Event table AccountCredited AccountDebited 101 450 Account table X 101 101 101 901 902 903 500 250 300

@crichardson Replay events to recreate state Account balance AccountOpenedEvent(balance) AccountDebitedEvent(amount) AccountCreditedEvent(amount) Events

@crichardson Aggregate traits Map Command to Events Apply event returning updated Aggregate

@crichardson Account - command processing Prevent overdraft

@crichardson Account - applying events Immutable

@crichardson Request handling in an event-sourced application HTTP Handler Event Store pastEvents = ﬁndEvents(entityId) Account new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A

@crichardson Event Store publishes events - consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate NoSQL materialized view update() update() Microservice B

@crichardson Event Store API trait EventStore { def save[T <: Aggregate[T]](entity: T, events: Seq[Event], assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]] def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion, entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]] def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) : Future[EntityWithIdAndVersion[T]] def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId) Future[Option[EntityWithIdAndVersion[T]]] def subscribe(subscriptionId: SubscriptionId): Future[AcknowledgableEventStream] } In case you are wondering: Akka Persistence is too Akka-centric

@crichardson Beneﬁts of event sourcing Business: Built in audit log Enables temporal queries Technical: Solves data consistency issues in a Microservice/NoSQL-based architecture: Atomically save and publish events Event subscribers update other aggregates ensuring eventual consistency Event subscribers update materialized views in SQL and NoSQL databases Eliminates O/R mapping problem

@crichardson Drawbacks of event sourcing Weird and unfamiliar Events = a historical record of your bad design decisions Handling duplicate events can be tricky Idempotent commands Duplicate detection, e.g. track most recently seen event

@crichardson The anatomy of a microservice Event Store HTTP Request HTTP Adapter Aggregate Event Adapter Cmd Cmd Events Events Xyz Adapter Xyz Request microservice

@crichardson Asynchronous Spring MVC controller

@crichardson AccountTransactionService DSL concisely speciﬁes: 1.Creates new TransferTransaction 2.Processes command 3.Applies events 4.Persists events

@crichardson TransferTransaction Aggregate

@crichardson Handling events published by Accounts 1.Load TransferTransaction 2.Processes command 3.Applies events 4.Persists events

@crichardson Let’s imagine that you want to display an account and it’s recent transactions...

@crichardson Displaying balance + recent transactions We need to do a “join: between the Account and the corresponding TransferTransactions (Assuming Debit/Credit events don’t include other account, ...) BUT Event Store = primary key lookup of individual aggregates, ... Use Command Query Responsibility Separation Deﬁne separate “materialized” query-side views that implement those queries

@crichardson Query-side microservices Event Store Updater - microservice View Updater Service Events Reader - microservice HTTP GET Request View Query Service View Store e.g. MongoDB Neo4J CloudSearch update query

@crichardson Persisting account balance and recent transactions in MongoDB { id: "298993498", balance: 100000, transactions : [ {"transactionId" : "4552840948484", "fromAccountId" : 298993498, "toAccountId" : 3483948934, "amount" : 5000}, ... ], changes: [ {"changeId" : "93843948934", "transactionId" : "4552840948484", "transactionType" : "AccountDebited", "amount" : 5000}, ... ] } Denormalized = efficient lookup Transactions that update the account The sequence of debits and credits

@crichardson Updating MongoDB using Spring Data class AccountInfoUpdateService (mongoTemplate : MongoTemplate, ...) extends CompoundEventHandler { @EventHandler def recordDebit(de: DispatchedEvent[AccountDebitedEvent]) = { ... val ci = AccountChangeInfo(...) mongoTemplate.updateMulti( new Query(where("id").is(de.entityId.id).and("version").lt(changeId)), new Update(). dec("balance", amount). push("changes", ci). set("version", changeId), classOf[AccountInfo]) } @EventHandler def recordTransfer(de: DispatchedEvent[TransferTransactionCreatedEvent]) = ... } insert/In-place update duplicate event detection updates to and from accounts

@crichardson Retrieving account info from MongoDB using Spring Data class AccountInfoQueryService(accountInfoRepository : AccountInfoRepository) { def findByAccountId(accountId : EntityId) : AccountInfo = accountInfoRepository.findOne(accountId.id) } case class AccountInfo(id : String, balance : Long, transactions : List[AccountTransactionInfo], changes : List[ChangeInfo], version : String) case class AccountTransactionInfo(changeId : String, transactionId : String, transactionType : String, amount : Long, balanceDelta : Long) trait AccountInfoRepository extends MongoRepository[AccountInfo, String] Implementation generated by Spring Data

@crichardson Building microservices with Spring Boot Makes it easy to create stand-alone, production ready Spring Applications Automatically configures Spring using Convention over Configuration Externalizes configuration Generates standalone executable JARs with embedded web server

@crichardson Spring Boot simplifies configuration Spring Container Application components Fully configured application Configuration Metadata •Typesafe JavaConfig •Annotations •Legacy XML Default Configuration Metadata Spring Boot You write less of this Inferred from CLASSPATH

@crichardson Tiny Spring conﬁguration for Account microservice @Configuration @EnableAutoConfiguration @Import(classOf[JdbcEventStoreConfiguration])) @ComponentScan class AccountConfiguration { @Bean def accountService(eventStore : EventStore) = new AccountService(eventStore) @Bean def accountEventHandlers(eventStore : EventStore) = EventHandlerRegistrar.makeFromCompoundEventHandler( eventStore, "accountEventHandlers", new TransferWorkflowAccountHandlers(eventStore)) @Bean @Primary def scalaObjectMapper() = ScalaObjectMapper } Service Event handlers Scan for controllers Customize JSON serialization

@crichardson The Main program object BankingMain extends App { SpringApplication.run(classOf[AccountConfiguration], args :_ *) }

@crichardson Building with Gradle buildscript { repositories { mavenCentral() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:1.0.0.RELEASE") } } apply plugin: 'scala' apply plugin: 'spring-boot' ...

@crichardson Running the microservice $ java -jar build/libs/banking-main-1.0-SNAPSHOT.jar --server.port=8081 ... 11:38:04.633 INFO n.c.e.e.bank.web.main.BankingMain$ - Started BankingMain. in 8.811 seconds (JVM running for 9.884) $ curl localhost:8081/health {"status":"UP", "mongo":{"status":"UP","version":"2.4.10"}, "db":{"status":"UP","database":"H2","hello":1} } Built in health checks Command line arg processing

@crichardson Jenkins-based deployment pipeline Build & Test microservice Build & Test Docker image Deploy Docker image to Repository One pipeline per micro-service

@crichardson Summary Event Sourcing solves key data consistency issues with: Microservices Partitioned/NoSQL databases Spring and Scala play nicely together Spring Boot makes it very easily to build production ready microservices

@crichardson Questions? @crichardson [email protected] http://plainoldobjects.com

Building microservices with Scala, functional domain models and Spring Boot

by Chris Richardson, Entrepreneur and software developer

on Aug 12, 2014

Statistics

Views

Actions

1 Embed 1

Accessibility

Categories

Upload Details

Usage Rights

Report content

Building microservices with Scala, functional domain models and Spring Boot Presentation Transcript