SEJ: Message Queue Based on Segmented Journals
h12.io/sej provides composable components of distributed, persisted message queue and allows trading off between reliablilty, latency and throughput with minimal devops overhead. Package Organization h12.io/sej: writer, scanner and offset shard: sharding hub: copying across machines cmd/sej: command line tool SEJ Directory [root-dir]/ [sej-dir]/ jnl.lck jnl/ 0000000000000000.jnl 000000001f9e521e.jnl ...... ofs/ reader1.ofs reader1.lck reader2.ofs reader2.lck ...... Journal File format segment_file = { message } . message = offset timestamp type key value size . offset = uint64 . timestamp = int64 . type = uint8 . key = key_size { uint8 } . key_size = int8 . value = value_size { uint8 } . value_size = int32 . size = int32 . All integers are written in the big endian format. ...