Inno Db Internals Inno Db File Formats And Source Code Structure

Transactional Storage for MySQL FAST. RELIABLE. PROVEN. InnoDB Internals: InnoDB File Formats and Source Code Structure MySQL Conference, April 2009 Heikki Tuuri Calvin Sun CEO Innobase Oy Principal Engineer Vice President, Development Oracle Corporation Oracle Corporation

Today’s Topics • Goals of InnoDB • Key Functional Characteristics • InnoDB Design Considerations • InnoDB Architecture • InnoDB On Disk Format • Source Code Structure • Q&A

Goals of InnoDB • OLTP oriented • Performance, Reliability, Scalability • Data Protection • Portability

InnoDB Key Functional Characteristics • Full transaction support • Row-level locking • MVCC • Crash recovery • Efficient IO

Design Considerations • Modeled on Gray & Reuter’s “Transactions Processing: Concepts & Techniques” • Also emulated the Oracle architecture • Added unique subsystems • Doublewrite • Insert buffering • Adaptive hash index • Designed to evolve with changing hardware & requirements

InnoDB Architecture Server Applications Handler API Embedded InnoDB API Transaction Cursor / Row Mini- B-tree Lock transaction Page Buffer File Space Manager IO

InnoDB On Disk Format • InnoDB Database Files • InnoDB Tablespaces • InnoDB Pages / Extents • InnoDB Rows • InnoDB Indexes • InnoDB Logs • File Format Design Considerations

InnoDB Database Files MySQL Data Directory System tablespace InnoDB tables internal data .frm files dictionary innodb_file_per_table OR insert buffer undo logs .ibd files ibdata files

InnoDB Tablespaces • A tablespace consists of multiple files and/or raw disk partitions. file_name:file_size[:autoextend[:max:max_file_size]] • A file/partition is a collection of segments. • A segment consists of fixed-length pages. • The page size is always 16KB in uncompressed tablespaces, and 1KB-16KB in compressed tablespaces (for both data and index).

System Tablespace • Internal Data Dictionary • Undo • Insert Buffer • Doublewrite Buffer • MySQL Replication Info

InnoDB Tablespaces Tablespace Segment Extent Extent Leaf node segment Non-leaf node segment Extent Extent Extent Rollback segment Page Row Row Row Trx id Row Row Row Roll pointer Row Row Field pointers Field 1 Field 2 Field n an extent = 64 pages

InnoDB Pages InnoDB Page Types Symbol Value Notes 3 File segment inode FIL_PAGE_INODE 17855 B-tree node FIL_PAGE_INDEX 10 Uncompressed BLOB page FIL_PAGE_TYPE_BLOB 1st compressed BLOB page 11 FIL_PAGE_TYPE_ZBLOB 12 Subsequent compressed BLOB page FIL_PAGE_TYPE_ZBLOB2 6 System page FIL_PAGE_TYPE_SYS 7 Transaction system page FIL_PAGE_TYPE_TRX_SYS i-buf bitmap, I-buf free list, file space others header, extent desp page, new allocated page

InnoDB Pages A page consists of: a page header, a page trailer, and a page body (rows or other contents). Page header Row Row Row Row Row Row Row Row Row Row Row row offset array Page trailer

Page Declares typedef struct /* a space address */ { ulint pageno; /* page number within the file */ ulint boffset; /* byte offset within the page */ } fil_addr_t; typedef struct { ulint checksum; /* checksum of the page (since 4.0.14) */ ulint page_offset; /* page offset inside space */ fil_addr_t previous; /* offset or fil_addr_t */ fil_addr_t next; /* offset or fil_addr_t */ dulint page_lsn; /* lsn of the end of the newest modification log record to the page */ PAGE_TYPE page type; /* file page type */ dulint file_flush_lsn;/* the file has been flushed to disk at least up to this lsn */ int space_id; /* space id of the page */ char data[]; /* will grow */ ulint page_lsn; /* the last 4 bytes of page_lsn */ ulint checksum; /* page checksum, or checksum magic, or 0 */ } PAGE, *PAGE;

InnoDB Compressed Pages • InnoDB keeps a “modification Page header log” in each page • Updates & inserts of small compressed data records are written to the log w/o page reconstruction; deletes don’t even require uncompression • Log also tells InnoDB if the modification log page will compress to fit page empty space size BLOB pointers • When log space runs out, InnoDB uncompresses the page directory page, applies the changes and Page trailer recompresses the page

InnoDB Rows … … prefix(768B) COMACT format overflow 20 bytes page … … DYNAMIC format overflow page Record hdr Trx ID Roll ptr Fld ptrs overflow-page ptr .. Field values

InnoDB Indexes - Primary rows are stored ●Data PK values 001 - nnn … … in the B-tree leaf nodes of a clustered index 001 – 500 – 801 – 500 800 nnn ●B-tree is organized xxx by primary key or - 501 631 769 801 950 001 276 – nnn - - - - - - 500 630 768 800 949 xxx 275 non-null unique key of table, if defined; clustered else, an internal (primary key) index Key values column with 6-byte 501- 501-630 Primary Index + data for corresponding rows ROW_ID is added.

InnoDB Indexes - Secondary clustered clustered (primary key) (primary PK values key) index - nnn 001 index ● Secondary index B- tree leaf nodes contain, for each key value, the primary B-tree leaf nodes, containing data keys of the corresponding rows, used to access key values AZ clustering index to obtain the data Secondary Index B-tree leaf nodes, containing PKs Secondary index Secondary index

InnoDB Logging Rollback segments Log Buffer Buffer Pool log thread write thread Log File Log File redo DATA #1 rollback #2 log log files ibdata files

InnoDB Redo Log end of log start of log last checkpoint min LSN Redo log structure: Space id PageNo OpCode Data

File Format Management • Builtin InnoDB format: “Antelope” • New “Barracuda” format enables compression,ROW_FORMAT=DYNAMIC • Fast index creation, other features do not .ibd require Barracuda file format data files (file per • Builtin InnoDB can access “Antelope” table) databases, but not “Barracuda” databases • Check file format tag in system tablespace on startup • Enable a file format with new dynamic parameter innodb_file_format • Preserves ability to downgrade easily

InnoDB File Format Design Considerations • Durability • Logging, doublewrite, checksum; • Performance • Insert buffering, table compression • Efficiency • Dynamic row format, table compression • Compatibility • File format management

Source Code Structure • 31 subdirectories • Relevant InnoDB source files on file formats • Tablespace: fsp0fsp {.c, .ic, .h} • Page: page0page, page0zip {.c, .ic, .h} • Log: log0log {.c, .ic, .h}

Source Code Subdirectories • buf • ibuf • que • data • include • read • db • lock • rem • dict • log • row • dyn • math • srv • eval • mem • sync • fil • mtr • thr • fsp • os • trx • fut • page • usr • ha • pars • ut • handler

Summary: Durability, Performance, Compatibility & Efficiency • InnoDB is the leading transactional storage engine for MySQL • InnoDB’s architecture is well-suited to modern, on- line transactional applications; as well as embedded applications. • InnoDB’s file format is designed for high durability, better performance, and easy to manage

For More Information … 2009 MySQL User Conference InnoDB Birds of a Feather Wed 7:30pm Ballroom C • Heikki Tuuri: Concurrency Control: How it Really Works, Thurs, 2:50pm Please visit www.innodb.com, blogs.innodb.com and forums.innodb.com

QUESTIONS ANSWERS

an company Plugin Hot Backup Embedded

InnoDB Size Limits Max # of tables: 4 G • Max size of a table: 32TB • Columns per table: 1000 • Max row size: n*4 GB • • 8 kB if stored on the same page • n*4 GB with n BLOBs • Max key length: 3500 • Maximum tablespace size: 64 TB • Max # of concurrent trxs: 1023

http://learnmysql.blogspot.com	50
http://feeds.feedburner.com	7
http://www.slideshare.net	6
http://learnvbdotnet.blogspot.com	3
http://localhost:3000	1

Inno Db Internals Inno Db File Formats And Source Code Structure

by MySQLConference on May 11, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Embed Views

Statistics

Inno Db Internals Inno Db File Formats And Source Code Structure — Presentation Transcript

Get analytics right now with SlideShare PRO

An effective way to find out who is viewing your content

Remove ads from all your pages with SlideShare PRO

Your viewers will engage with your content, not ads