Chronicle Map

  • Published on
    15-Apr-2017

  • View
    351

  • Download
    0

Embed Size (px)

Transcript

  • Chronicle Map

  • Chronicle Software

    Java

    ,

  • Chronicle Map

    Keyvalue store github.com/OpenHFT/Chronicle-Map

    LGPL v3

  • ?

  • (1)

    =

    Out-of-process (redis)

    Chronicle Map:

    (latency)

    < 1

  • (2)

    :

  • ?

  • NUMA

  • NUMA

  • NUMA

    ~30%

  • NUMA: 1

    NUMA node 1

    core core core

    NUMA node 2

    core core core

  • NUMA: 2

    NUMA node 1

    core core core

    NUMA node 2

    core core core

    1 2

    30% / 2 = 15%

  • JVM?

    heap

    Garbage-free

  • Compressed Oops

    JVM?

    Heap = 40 GB

    64-

    Heap = 15 GB

    32-

    Heap = 15 GB

    32-

  • JVM?

    Affinity

    cgroups

  • (2)

    Hazelcast

    IMDG

    MapDB

  • Chronicle Map:

    -

  • (3)

    :

    200k RPS

    3 KB state updates

    = 5 Gbit

  • (3)

    Chronicle Map:

    ,

  • Multikey ,

    durable, persistent

  • ?

  • ?

  • Loopback, sockets

    JVM 1 DB process

    JVM 2Kernel

    DB memory

  • Loopback, sockets

    : 100s of ns

    4

    2

    :

    1..10

  • Loopback, sockets

    Redis, Memcached

  • C + JNI

    Java

    sun.misc.Unsafe

  • C + JNI

    Java code DB code

    JNIJVM 1

    Kernel

    DB shared memory

    JVM 2

  • Java (Unsafe)

    Java codeDB code

    Unsafe

    JVM

    Kernel

    DB shared memory

    JVM 2

  • , Unsafe!

    JNI

    50 ns

    2

    putIfAbsent()

    Unsafe:

  • LockSupport.parkNanos(1000)

    8000

    Context switches

    Spin loops, CPU

  • :

    Chronicle Map*,

    one-nio*,

    ConcurrentHashMap

  • :orderId, instrument, price, amount

    + ,

    Key-value :instrument, price orderId, leftover

    4, 8 8, 8

    28

  • #1, AAPL, $600, +10

    AAPL, $600 #1, +10

    #2, AAPL, $600, 7

    AAPL, $600 #1, +3

    #3, AAPL, $600, 5

    AAPL, $600 #3, 2

  • 5000

    .

    :

  • 10 , 8

    ~ 160 .

    (instrument, price)

    Xeon E5-2650 v2 @ 2.60GHz

    8 Cores / 16 HW Threads,

    2 NUMA nodes, L3: 20MB

  • 174

    296

    367

    Chronicle Map one-nio ConcurrentHashMap

    ,

  • 7,5

    10,1

    14,4

    Chronicle Map one-nio ConcurrentHashMap

    , MB,

  • 7,5

    10,1

    14,4 15

    20

    ChronicleMap

    one-nio CHM redis 32-bit,approx.

    redis 64-bit,approx.

    , MB,

    Payload 4,5 MB

  • :

  • bit maps

    : ,

    eventually consistent

  • ffinity,

    syscalls, JNI

    Context switches

    IPC shared memory

  • http://chronicle.software/

    key-value-stories.blogspot.com

    @leventov

    leventov@ya.ru