23 October 2013 - AWS 201 - A Walk through the AWS Cloud: Introduction to Amazon Redshift

Amazon Redshi.

Security TECHNICAL CONTENT Backup Loading Demo Pricing Customers Use Cases Deﬁni1on Architecture TIME

Amazon RedshiN A fast and powerful, petabyte-‐scale data warehouse. Delivered as a managed service.

Where does it ﬁt?

Deployment & Administration Application Services Compute Storage Networking AWS Global Infrastructure Database

Amazon Elastic Map Reduce Amazon Redshift Data Warehouse Service Hosted Hadoop Service Amazon DynamoDB NoSQL Data Store Deployment & Administration Amazon RDS MySQL, Oracle and SQL Server Application Services Compute Storage Database Networking AWS Global Infrastructure Amazon S3 Object Storage

Structure Low High Large Hadoop (EMR) MPP DW (RedshiN) Size NoSQL (DynamoDB) Small Tradi1onal DW (RDS)

What are the beneﬁts?

•  Easy to provision and scale up massively •  Pay as you go •  Price-performance •  Standards-based

How are customers using it?

1. Replace ETL Application OLTP Database Data Warehouse Reporting and BI

1. Replace ETL Application OLTP Database Amazon Redshift Reporting and BI

2. Assist ETL Application OLTP Database Data Warehouse Reporting and BI

2. Assist Amazon Redshift Application Reporting and BI OLTP Database Data Warehouse

3. New Warehouse Application OLTP Database Reporting and BI

3. New Warehouse Application OLTP Database Amazon Redshift Reporting and BI

4. Log Analysis Amazon S3 Web / Application Servers Amazon Redshift Reporting and BI

Not Designed For: Transac1onal workload Very small data sets Sub-‐second response 1me

How does it work?

SQL Clients/BI Tools JDBC/ODBC 128GB RAM Leader Node 16 cores 16TB disk 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores 16 cores

SQL Clients/BI Tools ID Name 1 John Smith 2 Jane Jones 3 Peter Black 4 Pat Partridge 5 Sarah Cyan 6 Brian Snail JDBC/ODBC 128GB RAM Leader Node 16 cores 16TB disk 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores 16 cores 1 John Smith 2 Jane Jones 3 Peter Black 4 Pat Partridge 5 Sarah Cyan 6 Brian Snail

SQL Clients/BI Tools JDBC/ODBC 128GB RAM 16 cores Results SQL 16TB disk 128GB RAM 128GB RAM 16 cores 16 cores 128GB RAM 16 cores Results SQL Results SQL 16TB disk Results SQL 16TB disk 16TB disk 1 John Smith 2 Jane Jones 3 Peter Black 4 Pat Partridge 5 Sarah Cyan 6 Brian Snail

Do I have a choice of nodes?

16 GB RAM 2 cores 2 TB disk Compute Node Choice XL Single Node (2 TB) XL Cluster 2-‐32 Nodes (4 TB – 64 TB) XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL

128 GB RAM 16 cores 16 TB disk Compute Node Choice 8XL Cluster 2-100 Nodes (32 TB – 1.6 PB) 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL

How do I run queries?

JDBC/ODBC Query Tools Redshift DB Visualizer SQL Workbench

BI Tools

Do I need to performance tune?

Performance = Parallelism + Columnar + Compression + Zone Maps

SQL Clients/BI Tools JDBC/ODBC 128GB RAM Massively Parallel Processing Leader Node 16 cores 16TB disk 10 GigE 128GB RAM Choose Good Distribu1on Keys 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores 16 cores 1 John Smith 2 Jane Jones 3 Peter Black 4 Pat Partridge 5 Sarah Cyan 6 Brian Snail

ID State 123 20 CA 345 Data Storage: Age 25 WA 678 40 FL Row-‐based Vs Columnar Row storage Column storage

Raw encoding (RAW) Byte-‐dic1onary (BYTEDICT) Delta encoding (DELTA / DELTA32K) Compression Mostly encoding (MOSTLY8 / MOSTLY16 / MOSTLY32) Runlength encoding (RUNLENGTH) Text encoding (TEXT255 / TEXT32K) Average: 4-‐8x

CREATE TABLE orders ( orderkey int8 NOT NULL DISTKEY, custkey NOT NULL, orderstatus DDL int8 char(1) NOT NULL , totalprice numeric(12,2) NOT NULL , orderdate date NOT NULL SORTKEY , orderpriority char(15) NOT NULL, clerk char(15) NOT NULL , shippriority int4 NOT NULL, comment varchar(79) ); NOT NULL

How do I get data in?

S3 S3 Redshift copy events from 's3://mybucket/data/allevents_pipe.txt' credentials 'aws_access_key_id=<>; aws_secret_access_key=<>' maxerror 5 delimiter '|' timeformat 'YYYY-MM-DD HH:MI:SS';

DynamoDB DynamoDB Redshift copy favoritemovies from 'dynamodb://ProductCatalog' credentials 'aws_access_key_id=<>; aws_secret_access_key=<>' READRATIO 50;

Amazon Redshift ETL Tools Source Systems ETL

How do I back it up?

SQL Clients/BI Tools 128GB RAM Leader Node 16 cores •  •  Backup Automa1c Incremental 16TB disk 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores Amazon S3 16 cores

Can I stop a cluster?

SQL Clients/BI Tools 128GB RAM Leader Node 16 cores 16TB disk Snapshot 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores Snapshot Amazon S3 16 cores

SQL Clients/BI Tools 128GB RAM Leader Node 16 cores 16TB disk Snapshot Restore 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16TB disk 16TB disk 16TB disk 16 cores 16 cores Snapshot Amazon S3 16 cores

How do I resize it?

BI Tools 128GB RAM 128GB RAM Leader Node Leader Node 16 cores 16 cores 48TB disk 48TB disk Resize 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk

SQL Clients/BI Tools 128GB RAM Leader Node 16 cores 48TB disk Resize 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk 128GB RAM Compute Node 16 cores 48TB disk

Is it secure?

Customer VPC SQL Clients/BI Tools SSL Internal VPC 128GB RAM Leader Node 16 cores 16TB disk Encrypted Data at Rest Encrypted Data in Transit Security Groups DB Security 128GB RAM 128GB RAM 128GB RAM Compute Node Compute Node Compute Node 16 cores VPC 16TB disk 16 cores 16TB disk 16 cores 16TB disk Access Management SSL Amazon S3

How do I get started?

hop://aws.amazon.com/redshiN Detail FAQ Pricing Doco – Gepng Started Guide Forums Youtube.com Search for Amazon RedshiN Best Prac1ces

23 October 2013 - AWS 201 - A Walk through the AWS Cloud: Introduction to Amazon Redshift

by Amazon Web Services on Nov 01, 2013

Statistics

Views

Actions

0 Embeds 0

Accessibility

Categories

Upload Details

Usage Rights

Report content

23 October 2013 - AWS 201 - A Walk through the AWS Cloud: Introduction to Amazon Redshift Presentation Transcript