This book is about PySpark: Python API for Spark. Apache Spark is an analytics engine for large-scale data processing. Spark is the open source cluster computing system that makes data analytics fast to write and fast to run. This book provides a large set of recipes for implementing big data processing and analytics using Spark and Python. The goal of this book is to show working examples in PySpark so that you can do your ETL and analytics easier. You maycut and paste examples to deliver your applicationsin PySpark. This book introduces PySpark (Python API for Spark). You can use PySpark to tackle big datasets quickly through simple APIs in Python. You will learn how to express parallel tasks and computations with just a few lines of code, and cover applications from ETL,simple batch jobs to stream processing and machine learning. With this book, you may dive into Spark capabilities such as RDDs (resilient distributed datasets), dataframes (data as a table of rows and columns), in-memory caching, and the interactive PySpark shell, where you may leverage Spark's powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlib. In this book, you will learn Spark's transformations and actions by a set of well-defined and working examples.All examples are tested and working: this means that youcan copy-cut-paste to your desired PySpark applications.Writing PySpark is much easier than writing Spark applicationsin Java and PySpark applications are not bulky at all when compared to Java Spark. In this book you will learn: * Short introduction to Spark and PySpark * Learn about RDDs, DataFrames, SQL with worked examples * How to use important Spark transformations on RDDs (low-level APIs) * How to use SQL and DataFrame * How to read data from many different data sources and represent them as RDDs and DataFrames * Learn the power of Data Design Patterns * Learn the basics of Monoids and how you should use them in MapReduce * Learn the basics of GraphFrames for solving graph-related data problems * Implement Logistic Regression algorithms using PySpark * Basics of data partitioning and understand reduction transformations
Big Data Processing with Apache Spark Название: Big Data Processing with Apache Spark Автор: Srini Penchikala Издательство: Год: 2018 Страниц: 104 Формат: PDF Размер: 10 Mb Язык: English...
Data Analytics with Spark Using Python Название: Data Analytics with Spark Using Python Автор: Jeffrey Aven Издательство: Addison-Wesley Professional ISBN: 013484601X Год: 2018 ...
Big Data Processing Using Spark in Cloud Название: Big Data Processing Using Spark in Cloud Автор: Mamta Mittal Издательство: Springer Год: 2018 Страниц: 264 Формат: PDF, EPUB Размер: 12 Mb...
Learning Spark: Lightning-Fast Big Data Analysis Название: Learning Spark: Lightning-Fast Big Data Analysis Автор: Holden Karau, Andy Konwinski, Patrick Wendell Издательство: O'Reilly Media ISBN:...
Big Data Analytic with Spark Название: Big Data Analytic with Spark Автор: Mohammed Guller Издательство: Apress Год: 2015 Формат: PDF Размер: 10 Мб Язык: английский / English ...
Hadoop with Python Название: Hadoop with Python Автор: Zachary Radtka, Donald Miner Издательство: O'Reilly Год: 2015 Формат: pdf Страниц: 71 Размер: 1.75 MB Язык:...
Информация
Посетители, находящиеся в группе Гости, не могут оставлять комментарии к данной публикации.
Навигация
Вход на сайт
Реклама
PySpark Algorithms 
