SolutionsData

Realtime data sync

A change made in one system shows up everywhere else right away, not the next morning.

How we deliver it

  • We find which systems wait on a nightly batch, and what that costs you.
  • We publish changes as they happen and update every system that depends on them.
  • The nightly batch is retired once the live path is proven.

How it works

Instead of copying everything every night, each change is published once, as it happens, and delivered to every system that needs it. Those systems keep reading from the stores they already use; the changes simply arrive there sooner. The nightly batch is retired only once the live path has proven itself.

BEFOREnext dayAFTERsecondsChange inone systemWaits for thenightly batchOthers seeit tomorrowChange inone systemPublished asit happensOthers seeit in secondsBEFOREnext dayAFTERsecondsChange inone systemWaits for thenightly batchOthers seeit tomorrowChange inone systemPublished asit happensOthers seeit in seconds
One change, before and after.

One change, every system, its own shape

Each consuming system gets the data in the shape it needs, and none of them has to learn how the others work. When a source changes its structure, the processor in the middle absorbs it, so a dozen systems do not break at once.

CHANGEEVENT LOGSHAPED FOR EACHEACH SYSTEMA change,publishedonceEvent log,kept in orderIts ownschemaSearchAnalyticsPartnersystemA change,publishedonceEvent log,kept in orderIts ownschemaSearchAnalyticsPartnersystem
Published once, shaped and delivered for each system.

Nothing lost, nothing resent by hand

Changes are kept in order in an event log. A system that joins later reads it from the start and ends up with the current state; one that was down catches up on its own.

JOINS LATERreplays historyWAS DOWNcatches upNew systemReads the logfrom the startCurrentstateBack onlineReads whatit missedCurrentstateJOINS LATERreplays historyWAS DOWNcatches upNew systemReads the logfrom the startCurrentstateBack onlineReads whatit missedCurrentstate
New and recovering systems catch up from the log.

What decides it

How fresh it must be
Seconds matter for a customer who just updated their details, or a team acting on live changes. Where tomorrow is fine, a nightly batch is simpler and cheaper.
Who owns each piece of data
When a nightly batch and live changes both touch a record, each field needs one owner. That rule prevents conflicts rather than resolving them.
Which systems consume it
Each one is listed with the shape it needs. When a system only needs to be told about a change rather than receive the data, that is often an integration.
Where the systems run
Cloud services, your own servers, partners. Each receives changes over a path that suits it, without having to connect to the event log itself.

Hand over or run

Your choice, per solution. We hand it over with documentation your team can work from, or we keep it running: upgrades, monitoring and incidents, with one of us on call 24/7.

Proof

day → secondsportal change to searchable
a dozensystems fed from one stream
18 monthsin production, no downtime

Customer records that only changed on a nightly reload now reach a dozen systems, in the cloud and on premises, within seconds, without any of them switching databases.

Talk to the person who builds it

Tell us what is slow, stuck, or held together by a spreadsheet. We will tell you what we think in one call, before any proposal.

DanielCo-Founder

daniel@arendal.ca