Big Data Analysis with Python Programming

Introduction

There are many high-level languages. Python is one of the easiest languages to learn and use, while at the same time being very powerful. It is used by many of the most highly productive professional programmers. A few of the places that use Python extensively are Google, the New York Stock Exchange, Industrial Light and Magic. Python is used in different types of domains such as Big Data, Data Analysis, Rich Web and Internet Development, Game and 3D Graphics, Software Development, Database Access and so on.
Synopsis:This course will cover the installation, configuration, development and deployment of Python Programming Language and build Web Sites using the Django framework and Basic OpenERP. About 50% of the time will be instructor presentation and about 50% will be hands on labs.
How participants will benefit after the course:
• Learn to Design and develop web application in Django framework. 
• Very insightful for individuals willing to start with OpenERP development, which again, is developed in Django framework. It will also greatly assist to gain understanding of OpenERP architecture. 
• Learn to design database with proper planning and documentation. 
• Learn to do object oriented programming, network programming, XML programming, use different types of APIs and web services, unit testing, parser programming. 
• Very helpful for people working in the field of data analysis, Web mapping. 
• Learn to Design Principle, Version control, Project Management Method for Professional Work. 

Methodology

PowerPoint Presentation,Handouts,Hands on Lab Practice, Brainstorming

Contents of Training:

Session 1 :

An Introduction to Python and Basic Python Syntax:
Introductory about Python 
Installing Python 
Environment Variables
Basic Syntax
Comments
String Values and Operations
String Slices
String Operators
Numeric Data Types
Conversions
Simple Input and Output

Session 2 :

Language Components:
Control Flow and Syntax Indenting
The if Statement Relational Operators Logical Operators True or False
Bit Wise Operators The while Loop break and continue
The for Loop
Lists
Tuples
Sets
Dictionaries
Sorting Dictionaries
Copying Collections

Session 3 :
Functions:

Functions Parameters and Documentation
Keyword and Optional Parameters
Passing Collections to a Function
Variable Number of Arguments 
Scope
Lambda
Closures

Modules and Packages:

Modules and Packages 
Importing a module
Importing names
Directories as packages
Writing a module
Testing a module
Python debugger
Python profiler

Session 4:

Classes in Python:

Classes in Python
Principles of Object Orientation
Creating Classes
Instance Methods
File Organization
Class Variables 
Inheritance
Polymorphism 
Custom Exception Classes
Class Documentation – pydoc

Exceptions:

Errors
Run Time Errors 
The Exception Model
Exception Hierarchy

Session 5 :
Regular Expressions:

Introduction
Simple Character Matches 
Special Characters 
Character Classes
Quantifiers
The Dot Character
Greedy Matches
Grouping and Matching
Substituting
Splitting a String
Compiling Regular Expressions 

Implementing parsers in Python:

Overview of parsing, parsers and grammars 
Overview of finite state machines and their implementation using Python 
Overview of Python parser libraries 

XML - Python programming:

Overview of XML 
DTDs 
XML Namespaces 
Validating vs. nonvalidating XML 
Parsing XML using regular expressions 
Processing XML using SAX and DOM 

Session 6:

Python web programming and webservice and database with Python: 

Python CGI programming
Mod_python 
Using Python to generate HTML and XML pages 
Implementing simple HTTP servers using Python 
Overview of Web services 
Overview of Python based web services frameworks 
Overview of relational database concepts 
Overview of the Python database interface 
Connecting to MySQL from Python 

Session 7:

Python Django Framework and Getting Started with Django:

What is Django? 
Django and Python
Django’s take on MVC: Model, View and Template
DRY programming: Don’t Repeat Yourself
How to get and install Django
About the 3 Core Files: 
models.py
urls.py
views.py
Setting up database connections
Managing Users & the Django admin tool
Installing and using ‘out of the box’ Django features

Session 8 :

Django URL Patterns and Views and Forms:

Designing a good URL scheme
Generic Views
Form classes
Validation
Authentication
Advanced Forms processing techniques

Session 9:

Django & REST APIs and Unit Testing with Django:

Django REST framework
Django-piston
Overview / Refresher on Unit Testing and why it’s good
Using Python’s unittest2 library
Test
Test Databases

Session 10:

Introduction to Big Data

Big Data Concepts
Definition
Characteristics
Big Data vs. Relational Database Management Systems (RDBMS) 
Examples
Demonstration: Big Data vs. Relational Data

Big Data Analytics

Definition
Analytics Elements
Relationship of Analytics to Big Data
Examples
Demonstration: Convenience Store Purchases

Session 11:

Big Data Tools

Computation Engines

Overview
Characteristics
Use Cases
Examples (Hadoop, Google Compute Engine) 
Demonstration: Using Hadoop

Not Only SQL (NOSQL)

Overview
Characteristics
Use Cases
Comparison with Relational Database Management Tools
Comparison with Data Warehouse Tools
Examples (MongoDB, Couchbase, Cassandra, Oracle NoSQL Database Manager) 
Demonstration: Using MongoDB

Session 12:

Python for Hadoop’s MapReduce

Why this is important
Creating a map program
Creating a reduce program
How to do further analysis

Single Variable Analysis

Dot and jitter plots
Histograms and kernel density
Rank order plots
How to use summary statistic correctly