Big Data for Chimps. A Guide to Massive-Scale Data Processing in Practice

Spis treści ebooka

• Preface
  • What This Book Covers
  • Who This Book Is For
  • Who This Book Is Not For
  • What This Book Does Not Cover
  • Theory: Chimpanzee and Elephant
  • Practice: Hadoop
  • Example Code
  • A Note on Python and MrJob
  • Helpful Reading
  • Feedback
  • Conventions Used in This Book
  • Using Code Examples
  • Safari Books Online
  • How to Contact Us
• I. Introduction: Theory and Tools
• 1. Hadoop Basics
  • Chimpanzee and Elephant Start a Business
  • Map-Only Jobs: Process Records Individually
  • Pig Latin Map-Only Job
  • Setting Up a Docker Hadoop Cluster
    • Run the Job
  • Wrapping Up
• 2. MapReduce
  • Chimpanzee and Elephant Save Christmas
    • Trouble in Toyland
    • Chimpanzees Process Letters into Labeled Toy Forms
  • Pygmy Elephants Carry Each Toy Form to the Appropriate Workbench
  • Example: Reindeer Games
    • UFO Data
    • Group the UFO Sightings by Reporting Delay
    • Mapper
    • Reducer
    • Plot the Data
    • Reindeer Conclusion
  • Hadoop Versus Traditional Databases
  • The MapReduce Haiku
    • Map Phase, in Light Detail
    • Group-Sort Phase, in Light Detail
    • Reduce Phase, in Light Detail
  • Wrapping Up
• 3. A Quick Look into Baseball
  • The Data
  • Acronyms and Terminology
  • The Rules and Goals
  • Performance Metrics
  • Wrapping Up
• 4. Introduction to Pig
  • Pig Helps Hadoop Work with Tables, Not Records
    • Wikipedia Visitor Counts
  • Fundamental Data Operations
    • Control Operations
    • Pipelinable Operations
    • Structural Operations
  • LOAD Locates and Describes Your Data
    • Simple Types
    • Complex Type 1, Tuples: Fixed-Length Sequence of Typed Fields
    • Complex Type 2, Bags: Unbounded Collection of Tuples
    • Defining the Schema of a Transformed Record
  • STORE Writes Data to Disk
  • Development Aid Commands
    • DESCRIBE
    • DUMP
    • SAMPLE
    • ILLUSTRATE
    • EXPLAIN
  • Pig Functions
  • Piggybank
  • Apache DataFu
  • Wrapping Up
• II. Tactics: Analytic Patterns
• 5. Map-Only Operations
  • Pattern in Use
  • Eliminating Data
  • Selecting Records That Satisfy a Condition: FILTER and Friends
    • Selecting Records That Satisfy Multiple Conditions
    • Selecting or Rejecting Records with a null Value
    • Selecting Records That Match a Regular Expression (MATCHES)
      • Pattern in use
    • Matching Records Against a Fixed List of Lookup Values
      • Pattern in use
  • Project Only Chosen Columns by Name
    • Using a FOREACH to Select, Rename, and Reorder fields
      • Pattern in use
    • Extracting a Random Sample of Records
      • Pattern in use
    • Extracting a Consistent Sample of Records by Key
      • Pattern in use
    • Sampling Carelessly by Only Loading Some part- Files
    • Selecting a Fixed Number of Records with LIMIT
    • Other Data Elimination Patterns
  • Transforming Records
    • Transforming Records Individually Using FOREACH
    • A Nested FOREACH Allows Intermediate Expressions
    • Formatting a String According to a Template
    • Assembling Literals with Complex Types
      • Parsing a date
      • Assembling a bag
    • Manipulating the Type of a Field
    • Ints and Floats and Rounding, Oh My!
    • Calling a User-Defined Function from an External Package
  • Operations That Break One Table into Many
    • Directing Data Conditionally into Multiple Dataflows (SPLIT)
      • Demonstration in Pig
  • Operations That Treat the Union of Several Tables as One
    • Treating Several Pig Relation Tables as a Single Table (Stacking Rowsets)
  • Wrapping Up
• 6. Grouping Operations
  • Grouping Records into a Bag by Key
    • Pattern in Use
    • Counting Occurrences of a Key
      • Pattern in use
    • Representing a Collection of Values with a Delimited String
      • Pattern in use
    • Representing a Complex Data Structure with a Delimited String
      • Pattern in use
    • Representing a Complex Data Structure with a JSON-Encoded String
      • Pattern in use
      • Does God hate Cleveland?
  • Group and Aggregate
    • Aggregating Statistics of a Group
      • Pattern in use
    • Completely Summarizing a Field
      • Pattern in use
    • Summarizing Aggregate Statistics of a Full Table
      • Pattern in use
    • Summarizing a String Field
      • Pattern in use
  • Calculating the Distribution of Numeric Values with a Histogram
    • Pattern in Use
    • Binning Data for a Histogram
      • Histogram of career games played
    • Choosing a Bin Size
      • Bin size too large
      • Bin size too small
      • Bin size just right
    • Interpreting Histograms and Quantiles
      • Games played: linear
      • Games played: log-log plot
    • Binning Data into Exponentially Sized Buckets
      • Pattern in use
    • Creating Pig Macros for Common Stanzas
    • Distribution of Games Played
    • Extreme Populations and Confounding Factors
      • Distribution of birth and death day of year
      • Baseball player deaths
      • Baseball player births
    • Dont Trust Distributions at the Tails
    • Calculating a Relative Distribution Histogram
      • Pattern in use
    • Reinjecting Global Values
    • Calculating a Histogram Within a Group
      • Pattern in use
    • Dumping Readable Results
      • Pattern in use
  • The Summing Trick
    • Counting Conditional Subsets of a GroupThe Summing Trick
    • Summarizing Multiple Subsets of a Group Simultaneously
      • Pattern in use
    • Testing for Absence of a Value Within a Group
      • Pattern in use
  • Wrapping Up
  • References
• 7. Joining Tables
  • Matching Records Between Tables (Inner Join)
    • Joining Records in a Table with Directly Matching Records from Another Table (Direct Inner Join)
      • Disambiguating field names with ::
      • Body type versus slugging average
  • How a Join Works
    • A Join Is a COGROUP+FLATTEN
    • A Join Is a MapReduce Job with a Secondary Sort on the Table Name
      • Pattern in use
    • Handling nulls and Nonmatches in Joins and Groups
      • Pattern in use: inner join
  • Enumerating a Many-to-Many Relationship
  • Joining a Table with Itself (Self-Join)
  • Joining Records Without Discarding Nonmatches (Outer Join)
    • Pattern in Use
    • Joining Tables That Do Not Have a Foreign-Key Relationship
      • Pattern in use
    • Joining on an Integer Table to Fill Holes in a List
      • Pattern in use
  • Selecting Only Records That Lack a Match in Another Table (Anti-Join)
  • Selecting Only Records That Possess a Match in Another Table (Semi-Join)
    • An Alternative to Anti-Join: Using a COGROUP
  • Wrapping Up
• 8. Ordering Operations
  • Preparing Career Epochs
  • Sorting All Records in Total Order
    • Sorting by Multiple Fields
    • Sorting on an Expression (You Cant)
    • Sorting Case-Insensitive Strings
    • Dealing with nulls When Sorting
    • Floating Values to the Top or Bottom of the Sort Order
      • Pattern in use
  • Sorting Records Within a Group
    • Pattern in Use
    • Selecting Rows with the Top-K Values for a Field
    • Top K Within a Group
  • Numbering Records in Rank Order
    • Finding Records Associated with Maximum Values
    • Shuffling a Set of Records
  • Wrapping Up
• 9. Duplicate and Unique Records
  • Handling Duplicates
    • Eliminating Duplicate Records from a Table
    • Eliminating Duplicate Records from a Group
    • Eliminating All But One Duplicate Based on a Key
    • Selecting Records with Unique (or with Duplicate) Values for a Key
  • Set Operations
    • Set Operations on Full Tables
    • Distinct Union
    • Distinct Union (Alternative Method)
    • Set Intersection
    • Set Difference
    • Symmetric Set Difference: (AB)+(BA)
    • Set Equality
    • Set Operations Within Groups
    • Constructing a Sequence of Sets
    • Set Operations Within a Group
  • Wrapping Up
• Index

pokaż cały spis treści

ukryj recenzje