Amazon Relational Database Service (Amazon RDS)

  • Published on
    13-Feb-2017

  • View
    232

  • Download
    15

Embed Size (px)

Transcript

<ul><li><p>Amazon Aurora</p><p>AWS Black Belt Online Seminar 2016</p></li><li><p>AWS Black Belt Online Seminar AWSJTechAWS</p><p> 18:00~19:00AWS(EC2RDSLambda etc.)</p><p> 12:00~13:00AWS(IoT etc.)</p><p>&amp; http://aws.amazon.com/jp/about-aws/events/#webinar</p><p>http://aws.amazon.com/jp/about-aws/events/#webinar</p></li><li><p>2016729AWS(http://aws.amazon.com/ )</p><p>AWSAWS</p><p>AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.</p><p>http://aws.amazon.com/</p></li><li><p>2016/7/29</p></li><li><p>Amazon Aurora</p></li><li><p> S3</p><p> DBA </p><p>Amazon RDS</p></li><li><p>Virginia / Oregon / Ireland / Sydney / Tokyo / Seoul / Mumbai</p></li><li><p>Amazon Aurora</p><p>MySQL5.6</p></li><li><p> 1970</p><p> AWS</p></li><li><p>MySQL</p><p>Business Intelligence Data Integration Query and Monitoring SI and Consulting</p><p>Source: Amazon</p><p>Amazon Aurora- Dan Jewett, Vice President of Product Management at Tableau</p></li><li><p>8GB</p><p>16GB</p><p>32GB</p><p>60GB</p><p>122GB</p><p>244GB</p><p>4core 8core 16core 32core</p><p>r3.8xl</p><p>2core1core</p><p>r3.4xl</p><p>r3.2xl</p><p>r3.xl</p><p>r3.large</p><p>4GB</p><p>1GB</p><p>Mem</p><p>ory</p><p>vCPU</p></li><li><p>Service Oriented Architecture</p><p> EC2, Amazon DynamoDB, Amazon SWFAWS</p><p> Amazon S399.999999999%</p><p>Data Plane</p><p>Logging + Storage</p><p>SQL</p><p>Transactions</p><p>Caching</p><p>Amazon S3</p><p>Control Plane</p><p>Amazon DynamoDB</p><p>Amazon SWF</p><p>Amazon Route 53</p></li><li><p> + = DB</p><p>SQL</p><p>Transactions</p><p>Caching</p><p>SQL</p><p>Transactions</p><p>Caching</p><p>SQL</p><p>Transactions</p><p>Caching</p><p>DBDB</p></li><li><p> AES-256 ()</p><p> Amazon S3</p><p> AWS KMS</p><p> SSL</p><p> Amazon VPC</p><p>Storage</p><p>SQL</p><p>Transactions</p><p>Caching</p><p>Amazon S3</p><p>Application</p></li><li><p>Aurora</p><p> SSD</p><p> Log structured Storage</p><p>SQL</p><p>Transactions</p><p>AZ 1 AZ 2 AZ 3</p><p>Caching</p><p>Amazon S3</p></li><li><p>Aurora</p><p> S3 </p><p> 64TB </p></li><li><p> 2 3 </p><p>SQL</p><p>Transaction</p><p>AZ 1 AZ 2 AZ 3</p><p>Caching</p><p>SQL</p><p>Transaction</p><p>AZ 1 AZ 2 AZ 3</p><p>Caching</p></li><li><p>IO traffic in Aurora ()</p><p>LOG RECORDS</p><p>Primary </p><p>instance</p><p>INCOMING QUEUE</p><p>STORAGE NODE</p><p>S3 BACKUP</p><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p><p>6</p><p>7</p><p>8</p><p>UPDATE </p><p>QUEUE</p><p>ACK</p><p>HOT</p><p>LOG</p><p>DATA</p><p>BLOCKS</p><p>POINT IN TIME</p><p>SNAPSHOT</p><p>GC</p><p>SCRUB</p><p>COALESCE</p><p>SORT</p><p>GROUP</p><p>PEER-TO-PEER GOSSIPPeer</p><p>storage</p><p>nodes</p><p>1 2</p><p>MySQL1/46 (unamplified, per node)</p><p>OBSERVATIONS</p><p>IO FLOW</p><p> SSDACK </p><p> S3</p><p> CRC</p></li><li><p>Aurora Master</p><p>30% Read</p><p>70% Write</p><p>Aurora Replica</p><p>100% New </p><p>Reads</p><p>Shared Multi-AZ Storage</p><p>MySQL Master</p><p>30% Read</p><p>70% Write</p><p>MySQL Replica</p><p>30% New Reads</p><p>70% Write</p><p>Binlog</p><p>Data Volume Data Volume</p><p>MySQL read scaling</p><p> binlog / relay log</p><p>PAGE CACHE</p><p>UPDATE</p></li><li><p>: Aurora vs. RDS for MySQL</p><p>Primary</p><p>r3.8XL</p><p>Standby</p><p>r3.8XL</p><p>Replica</p><p>r3.8XLReplica</p><p>R3.8XL</p><p>Storage6 TB / 10 K PIOP</p><p>Storage6 TB / 10 K PIOP</p><p>Storage6 TB / 5 K PIOP</p><p>Storage6 TB / 5 K PIOP</p><p>$1.33/hr</p><p>$1.33/hr</p><p>$1.33/hr $1.33/hr</p><p>$2,42/hr</p><p>$2,42/hr $2,42/hr</p><p>Instance cost: $5.32 / hr</p><p>Storage cost: $8.30 / hr</p><p>Total cost: $13.62 / hr</p><p>$2,42/hr</p></li><li><p> : Aurora vs. RDS for MySQL</p><p>Instance cost: $4.86 / hr</p><p>Storage cost: $4.43 / hr</p><p>Total cost: $9.29 / hr</p><p>Primary</p><p>r3.8XL</p><p>Replica</p><p>r3.8XL</p><p>Replica</p><p>R3.8XL</p><p>Storage / 6 TB</p><p>$1.62 / hr $1.62 / hr $1.62 / hr</p><p>$4.43 / hr</p><p>31.8%</p><p>Savings</p><p> No POIPs pay for use I/O</p></li><li><p> Throughput Select Commit DML/DDL</p><p> Latency Select Commit DML/DDL</p><p> Cache Hit Ratio Buffer Cache Result Set</p><p> Deadlocks Login Failures Blocked Transactions</p></li><li><p> IOPS Billed storage</p><p> Billed read operations</p><p> Billed write operations</p></li><li><p> 1 RDS for MySQL</p><p> 15</p><p> Multi-AZAZ RDS for MySQL</p></li><li><p> WriterReaderWriter()</p><p> Aurora(Endpoint)</p><p>Reader</p><p>Writer</p></li><li><p>Availability Zone A Availability Zone B</p><p>VPC subnet VPC subnet</p><p>VPC subnet VPC subnet</p><p>Aurora Writer Aurora Reader</p><p> Aurora</p><p> Aurora WriterCNAME</p><p> ReadReader</p><p>Write</p></li><li><p> Aurora</p><p>Availability Zone A Availability Zone B</p><p>VPC subnet VPC subnet</p><p>VPC subnet VPC subnet</p><p>Aurora Writer Aurora Writer</p><p>Write</p></li><li><p> AuroraRDS MySQL</p><p> WriterReaderReaderAurora</p><p> Reader</p></li><li><p>ApprunningFailure detection DNS propagation</p><p>Recovery Recovery</p><p>DBfailure</p><p>MySQL</p><p>App</p><p>running</p><p>Failure detection DNS propagation</p><p>Recovery</p><p>DB</p><p>failure</p><p>AURORA WITH MARIADB DRIVER</p><p>1 5 - 2 0 s e c</p><p>3 - 2 0 s e c</p></li><li><p>Streaming snapshotPITR</p><p> Amazon AuroraAmazon S3 Backup retention period</p><p> Amazon Aurora</p><p> PITR5Backup Retention Period</p></li><li><p> MySQL</p><p>Amazon Aurora</p><p> Disk readredo log</p><p>Checkpointed Data Redo Log</p><p>T0 </p><p>T0 T0</p><p>T0 redo</p></li><li><p>SQL</p><p>SQL</p><p> :ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]</p><p> :ALTER SYSTEM SIMULATE percentage_of_failure PERCENT </p><p>READ REPLICA FAILURE [ TO ALL | TO "replica name" ]FOR INTERVAL quantity [ YEAR | QUARTER | MONTH | WEEK| DAY | HOUR | </p><p>MINUTE | SECOND ];</p></li><li><p>WRITE PERFORMANCE READ PERFORMANCE</p><p>MySQL SysBench results</p><p>R3.8XL: 32 cores / 244 GB RAM</p><p>5X faster than RDS MySQL 5.6 &amp; 5.7</p><p>SysBench</p><p>MySQL5</p><p>0</p><p>25,000</p><p>50,000</p><p>75,000</p><p>100,000</p><p>125,000</p><p>150,000</p><p>0</p><p>100,000</p><p>200,000</p><p>300,000</p><p>400,000</p><p>500,000</p><p>600,000</p><p>700,000</p><p>Aurora MySQL 5.6 MySQL 5.7</p></li><li><p>WRITE PERFORMANCE READ PERFORMANCE</p><p>AuroraRead/Write</p><p>Aurora MySQL 5.6 MySQL 5.7</p></li><li><p>Beyond benchmarks </p><p> POSSIBLE DISTORTIONS</p><p> HA</p></li><li><p> Amazon AuroraMySQL</p><p> CPU</p><p> Amazon Aurora </p><p> buffer poolcache hit</p></li><li><p> Amazon Aurora</p></li><li><p>I/O</p><p>DO LESS WORK</p><p>BE MORE EFFICIENT</p><p> I/O </p><p> PACKETS/SECOND </p></li><li><p>IO traffic in RDS MySQL</p><p>BINLOG DATA DOUBLE-WRITELOGFRM </p><p>FILES</p><p>T Y P E O F W R I T E</p><p>MYSQL WITH STANDBY</p><p>EBS EBSack</p><p>EBS</p><p>IO FLOW</p><p>1, 3, 5</p><p>2</p><p>OBSERVATIONS</p><p>780K </p><p>1007,388K I/Os (, )</p><p>17.4 I/Os</p><p>PERFORMANCE</p><p>30 minute SysBench write-only workload, 100 GB data set, RDS SingleAZ, 30K </p><p>PIOPS</p><p>EBS mirrorEBS mirror</p><p>AZ 1 AZ 2</p><p>Amazon S3</p><p>EBSAmazon Elastic </p><p>Block Store (EBS)</p><p>Primary</p><p>instance</p><p>Standby</p><p>instance</p><p>1</p><p>2</p><p>3</p><p>4</p><p>5</p></li><li><p>IO traffic in Aurora ()</p><p>AZ 1 AZ 3</p><p>Primary</p><p>instance</p><p>Amazon S3</p><p>AZ 2</p><p>Replica</p><p>instance</p><p>AMAZON AURORA</p><p>ASYNC</p><p>4/6 QUORUM</p><p>DISTRIBUTED </p><p>WRITES</p><p>BINLOG DATA DOUBLE-WRITELOG FRM </p><p>FILES</p><p>T Y P E O F W R I T E</p><p>30 minute SysBench write-only workload, 100 GB data set</p><p>IO FLOW</p><p>REDO; </p><p>(, )</p><p>6, 1/9</p><p>OBSERVATIONS</p><p>27,378K 35X MORE</p><p>100950K I/Os 7.7X LESS(6X amplification)</p><p>PERFORMANCE</p><p>REDO LSN</p></li><li><p> Amazon Aurora MariaDBMySQL EEAmazon </p><p>Aurora </p><p>ClientThread</p><p>Thread</p><p>Thread</p><p>Thread</p><p>Client</p><p>MySQL</p><p>ClientThread</p><p>Thread</p><p>Thread</p><p>Thread</p><p>Client</p><p>Aurora</p><p>Thread Pool</p></li><li><p> r3.8xlAmazon Aurora5,000</p><p> Standard MySQL 1</p><p> MySQL EE </p><p>CLIE</p><p>NT</p><p> CO</p><p>NN</p><p>EC</p><p>TIO</p><p>N</p><p>CLIE</p><p>NT</p><p> CO</p><p>NN</p><p>EC</p><p>TIO</p><p>N</p><p>LATCH FREE</p><p>TASK QUEUE</p><p>ep</p><p>oll(</p><p>)</p><p>MYSQL THREAD MODEL AURORA THREAD MODEL</p><p>Adaptive Thread Pool</p></li><li><p>Read</p><p>Write</p><p>Commit</p><p>Read</p><p>Read</p><p>T1</p><p>Commit (T1)</p><p>Commit (T2)</p><p>Commit (T3)</p><p>LSN 10</p><p>LSN 12</p><p>LSN 22</p><p>LSN 50</p><p>LSN 30 </p><p>LSN 34</p><p>LSN 41</p><p>LSN 47</p><p>LSN 20</p><p>LSN 49</p><p>Commit (T4)</p><p>Commit (T5)</p><p>Commit (T6)</p><p>Commit (T7)</p><p>Commit (T8)</p><p>LSN GROWTHDurable LSN at head-node </p><p>COMMIT QUEUEPending commits in LSN order</p><p>TIME</p><p>GROUP</p><p>COMMIT</p><p>TRANSACTIONS</p><p>Read</p><p>Write</p><p>Commit</p><p>Read</p><p>Read</p><p>T1</p><p>Read</p><p>Write</p><p>Commit</p><p>Read</p><p>Read</p><p>Tn</p><p>TRADITIONAL APPROACHAMAZON AURORA</p><p>I/O</p><p>64ACK</p></li><li><p>read/write I/O</p><p>malloc</p><p>Undo</p><p>binlog</p><p>(read-ahead)</p><p>read/write</p><p>mutex</p></li><li><p>Amazon Aurora</p></li><li><p> Amazon Aurora CPU</p><p> Query Cache</p></li><li><p> DB1 </p></li><li><p>MySQL</p><p> Percona XtrabackupAmazon EC2MySQLAmazon Aurora</p><p> mysqldump20</p><p> S3 Management ConsoleCLI tools</p><p>AWS Import/Export SnowballS3</p><p> MySQLAmazon Aurora</p></li><li><p> WriterReader </p><p> API</p><p> DRDB</p><p> DB</p><p>binlog_format(MIXED) </p></li><li><p>Aurora</p><p> Aurora AWS()</p><p> AWSAurora</p></li><li><p>AuroraSnapshot</p><p> AuroraKMSAurora</p><p> Snapshotrestore</p><p> SnapshotrestoreEnable EncryptionYesKey</p></li><li><p> time_zone RDS MySQL, RDS MariaDB</p></li><li><p> Amazon Aurora Reader</p><p> ReaderReader</p><p> : Tier 0 &gt; Tier 1 &gt; &gt; Tier 15</p><p> Reader Writer</p><p> Reader</p></li><li><p>Cluster View</p><p> Amazon Aurora Cluster Cluster</p><p> : Cluster SnapshotClusterDBClusterInstance ViewCluster View</p></li><li><p>Process list</p><p>Metrics list</p></li><li><p>OS</p><p>User </p><p>System </p><p>Wait </p><p>IRQ </p><p>Idle </p><p>CPU Utilization </p><p>Rx per declared ethn</p><p>Tx per declared ethn</p><p>Network</p><p>Num processes </p><p>Num interruptible</p><p>Num non-interruptible </p><p>Num zombie </p><p>Processes</p><p>Process ID </p><p>Process name </p><p>VSS </p><p>Res </p><p>Mem % </p><p>consumed </p><p>CPU % used </p><p>CPU time </p><p>Parent ID</p><p>Process List</p><p>MemTotal</p><p>MemFree</p><p>Buffers </p><p>Cached </p><p>SwapCached</p><p>Active </p><p>Inactive </p><p>SwapTotal</p><p>SwapFree</p><p>Dirty </p><p>Writeback</p><p>Mapped </p><p>Slab </p><p>MemoryTPS </p><p>Blk_read</p><p>Blk_wrtn</p><p>read_kb</p><p>read_IOs</p><p>read_size</p><p>write_kb</p><p>write_IOs</p><p>write_size</p><p>avg_rw_size</p><p>avg_queue_len</p><p>Device IO</p><p>Free </p><p>capacity </p><p>Used </p><p>% Used </p><p>File System</p></li><li><p> CloudWatch LogsElasticsearch Service</p></li><li><p>Encryption at Rest</p><p> Key Management Service(KMS) Aurora</p><p> Snapshot</p><p> SnapshotAurora</p><p> Disk</p><p> (KMS)</p></li><li><p> Large dataset read performance</p><p> IO/CPUAuroraIO/CPU</p><p> Fast Insert Primary key LOAD DATA INSERT INTO ... SELECT </p><p> () Global</p><p> aurora_fast_insert_cache_hits: cursor aurora_fast_insert_cache_misses: index</p><p> Parallel Read Ahead B-TreeDisk page</p><p> WriterReader </p></li><li><p>Lab mode</p><p> DB aurora_lab_mode </p><p> Production</p><p> OFF</p></li><li><p>Amazon Aurora</p><p> AmazonRDBMS MySQL5.6</p><p> Amazon Aurora</p></li><li><p>Webinar</p><p> AWS </p><p> http://aws.amazon.com/jp/aws-jp-introduction/</p><p> AWS Solutions Architect Q&amp;A</p><p> http://aws.typepad.com/sajp/</p><p>http://aws.amazon.com/jp/aws-jp-introduction/http://aws.typepad.com/sajp/</p></li><li><p>Q&amp;A</p><p>[]</p><p>http://aws.amazon.com/jp/contact-us/aws-sales/</p><p>[]</p><p>https://aws.amazon.com/jp/contact-us/</p><p>http://aws.amazon.com/jp/contact-us/aws-sales/https://aws.amazon.com/jp/contact-us/</p></li><li><p>Webinar</p><p> AWS http://aws.amazon.com/jp/aws-jp-introduction/</p><p>http://aws.amazon.com/jp/aws-jp-introduction/</p></li><li><p>Twitter/FacebookAWS</p><p>@awscloud_jp</p><p>http://on.fb.me/1vR8yWm</p></li></ul>

Recommended

View more >