An introduction to Apache S4

 Apache S4  What is it ?

●

 Building Blocks

●

 Characteristics

●

 Architecture

●

 Why was it needed ?

●

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – What is it ?  S4 = SSSS

●

●

Simple Scalable Streaming System

 For data processing

●

 Aims to hide processing compleity

●

 A plat!orm !or data processing de"elopment

●

 #istributed

●

 Scalable

●

 Fault $olerant

●

 %igh per!ormance

●

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – What is it ?  &t is modular 

●

 Written in 'a"a

●

 Currently in the Apache incubator 

●

 (!!ers plug)ins

●

 For comple data stream processing

●

 *roduction deployment at +ahoo

●

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – Building Blocks What are the S4 building blocks ?  –

,essage -ueues

 –

*rocessors

 –

Seriali.er 

 –

Checkpointing backend

*lus  –

 A simple A*&

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – Characteristics  Cluster management uses /oo 0eeper 

●

 Symmetric nodes

●

 –

1o centrali.ation

 –

1o single point o! !ailure

 1o limit on nodes

●

 –

$hroughput increases linearly

 Fault tolerant !ailo"er to standby ser"er 

●

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – Architecture

www.semtech-solutions.co.nz   [email protected]

 Apache S4 – Why was in needed needed ?

www.semtech-solutions.co.nz   [email protected]

Contact 2s  Feel !ree to contact us at

●

 –

www3semtech)solutions3co3n.

 –

in!osemtech)solutions3co3n.

 We o!!er &$ pro5ect consultancy

●

 We are happy to hear about your problems

●

 +ou can 5ust pay !or those hours that you need

●

 $o sol"e your problems

●