Yard -yet another resource dispatcher

  • Published on
    21-Aug-2015

  • View
    54

  • Download
    11

Embed Size (px)

Transcript

<ol><li> 1. YARD </li><li> 2. YARD YARD Yet Another Resource Dispatcher </li><li> 3. YARD </li><li> 4. YARD CPU API YARD = BTAgent + Master + API YARD = BTAgent + Master + API </li><li> 5. YARD API svrkit YARD YARD MapReduce Spark Docker </li><li> 6. YARD </li><li> 7. YARD Master </li><li> 8. </li><li> 9. Map Reduce Spark Docker SDN VLAN </li><li> 10. YARD YARD YARD </li><li> 11. 1 IDC CPU Core GB 1794 16997 35930 835 8089 19467 89 994 2541 2718 26080 57938 * 40Mb/s CPU 25% 60% * B C KV C </li><li> 12. 2 IDC CPU Core GB 2667 23564 50276 1097 10200 24412 131 1372 3392 3895 35136 78080 * 100Mb/s CPU 45% 60% * B C KV C </li><li> 13. Application Master AM DAG </li><li> 14. YARD Zookeeper Master (standby) Master (main) Redis Redis loser Zookeeper Master (main) Master (standby) winner Redis Redis Machine Slave Application Master Master Machine Slave Application Master Machine Slave Application Master Master Zookeeper Executor Executor Executor Zookeeper Executor Executor Executor Machine Slave Executor Application Master Machine Slave Executor Machine Slave Executor Application Master Zookeeper Zookeeper Task Task Task </li><li> 15. YARD API </li><li> 16. DataCenter OS UCBerkeley: The Datacenter Needs an Operating System Mesosphere: Datacenter Operating System (DCOS) </li><li> 17. DataCenter OS Resource sharing Lower-level interfaces for fine-grained sharing (Mesos is a first step in this direction) Optimization for a variety of metrics (e.g. energy) Integration with network scheduling mechanisms (e.g. Seawall [NSDI 11], NOX, Orchestra) </li><li> 18. DataCenter OS Data sharing Standard interfaces for cluster file systems, key-value stores, etc Programming abstractions Tools that can be used to build the next MapReduce / BigTable in a week (e.g. BOOM) Debugging facilities Tracing and debugging tools that work across the cluster software stack (e.g. X-Trace, Dapper) </li></ol>