IBM Systems Magazine, Mainframe - November/December 2015 - 19

IBM brings the Apache project's analytics power
to Linux and z Systems servers
By Mythili Venkatakrishnan k Illustration by Paul Price

A

nalytics is increasingly
an integral part of dayto-day operations at
today's leading businesses, as
evidenced by the 2015 Gartner
Executive Summary of the CIO
Agenda (bit.ly/1Etr6qs). Eighty
percent of CIOs surveyed stated
that transition from backwardlooking, passive analysis
must shift to forward-looking
predictive analytics and active
experimentation. Transformation
is also occurring through huge
growth in mobile and digital
channels. The same survey
shows that whereas separating
analytics and operations had
been acceptable in the past, 62
percent of CIOs now feel the
need to shift to a more embedded
analytics environment.
Previously acceptable response
times and delays for analytic
insight are no longer viable, with
a stronger push toward real-time
and in-transaction analytics.
In addition, data science skills
are increasingly in demand.
One McKinsey Global Institute
report (bit.ly/1LFPJjF) states

that by 2018, the U.S. could
face a shortage of as much as
190,000 people with data science
skills. As a result, enterprise
organizations are attempting to
leverage analytics in new ways
and transition existing analytic
capabilities to respond with more
flexibility while making the most
efficient use of highly valuable
data science skills.
Though the demand for
more agile analytics across the
enterprise is increasing, many
of today's solutions are aligned
to specific platforms, tied to
inflexible programming models,
require vast data movements into
data lakes that quickly become
data swamps, and result in
pockets of analytics and insight
that require ongoing manual
intervention to integrate into
coherent analytics solutions.
With all of these impending
forces converging, organizations are
well poised for a change. The recent
growth and adoption of Apache
Spark as an analytics framework
and platform is timely and helps
meet these challenging demands.

What is Apache Spark?

D

IBM plans to put
more than

3,500

IBM developers
and researchers to work
on worldwide
Spark-related
projects

Spark is an open source, in-memory
analytics computing framework
offered by the Apache Foundation.
Spark offers a unified programming environment and is
extremely lightweight. Most importantly it's function-rich, meaning
it provides libraries for commonly
used analytic methodologies for
data access, manipulation and
application of various algorithms.
Spark offers language diversity in
its support for Java*, Python, Scala
and, most recently, R.
From an operations point of
view, Spark can run in stand-alone
mode or clustered environments,
and it's not reliant on a specific
file system or platform set of
technologies, but it can be adapted
to many configurations.
In June, IBM announced a
major commitment to Spark,
including plans to put more
than 3,500 IBM developers and
researcher to work on Sparkrelated projects worldwide,
contribution of the IBM SystemML
machine learning technology to
the Spark open-source community

ibmsystemsmag.com NOVEMBER/DECEMBER 2015 // 19


http://www.bit.ly/1Etr6qs http://www.bit.ly/1LFPJjF http://www.ibmsystemsmag.com

IBM Systems Magazine, Mainframe - November/December 2015

Table of Contents for the Digital Edition of IBM Systems Magazine, Mainframe - November/December 2015

IBM Systems Magazine, Mainframe - November/December 2015
Table of Contents
IBM Perspective: Opening new options for real-time analytics
Editor's Desk: Predictive analytics enhances purchases
Infographic: Breaking down the mainframe skills gap
Partner PoV: IBM z Systems clients must manage existing and emerging assets together to get the most value
Focus on Storage: The IBM Spectrum Storage family of software-defined solutions simplifies storage management
Currents: Architect real-time analytics into operational systems for competitive advantage
Cover Story: Dynamic Spark: IBM brings the Apache project's analytics power to Linux and z Systems servers
Feature: Personal Shopper: Retailer optimizes analytics to drive a more customized client experience
Tech Showcase: In a roundtable, distributors discuss how IBM LinuxONE enables flexibility for users
Tech Corner: Runtime Diagnostics and Predictive Failure Analysis V2.2 improve z/OS availability
Solutions: Strobe 5.2
Stop Run: IBMer volunteers time to educate people about malaria and technology
Reference Point - Global Events, Education, Resources for Mainframe
2016 Mainframe Solutions Edition Product Index
IBM Systems Magazine, Mainframe - November/December 2015 - Intro
IBM Systems Magazine, Mainframe - November/December 2015 - IBM Systems Magazine, Mainframe - November/December 2015
IBM Systems Magazine, Mainframe - November/December 2015 - Cover2
IBM Systems Magazine, Mainframe - November/December 2015 - 1
IBM Systems Magazine, Mainframe - November/December 2015 - Table of Contents
IBM Systems Magazine, Mainframe - November/December 2015 - 3
IBM Systems Magazine, Mainframe - November/December 2015 - 4
IBM Systems Magazine, Mainframe - November/December 2015 - IBM Perspective: Opening new options for real-time analytics
IBM Systems Magazine, Mainframe - November/December 2015 - Editor's Desk: Predictive analytics enhances purchases
IBM Systems Magazine, Mainframe - November/December 2015 - 7
IBM Systems Magazine, Mainframe - November/December 2015 - Infographic: Breaking down the mainframe skills gap
IBM Systems Magazine, Mainframe - November/December 2015 - Partner PoV: IBM z Systems clients must manage existing and emerging assets together to get the most value
IBM Systems Magazine, Mainframe - November/December 2015 - 10
IBM Systems Magazine, Mainframe - November/December 2015 - Focus on Storage: The IBM Spectrum Storage family of software-defined solutions simplifies storage management
IBM Systems Magazine, Mainframe - November/December 2015 - 12
IBM Systems Magazine, Mainframe - November/December 2015 - 13
IBM Systems Magazine, Mainframe - November/December 2015 - 14
IBM Systems Magazine, Mainframe - November/December 2015 - Currents: Architect real-time analytics into operational systems for competitive advantage
IBM Systems Magazine, Mainframe - November/December 2015 - 16
IBM Systems Magazine, Mainframe - November/December 2015 - 17
IBM Systems Magazine, Mainframe - November/December 2015 - Cover Story: Dynamic Spark: IBM brings the Apache project's analytics power to Linux and z Systems servers
IBM Systems Magazine, Mainframe - November/December 2015 - 19
IBM Systems Magazine, Mainframe - November/December 2015 - 20
IBM Systems Magazine, Mainframe - November/December 2015 - 21
IBM Systems Magazine, Mainframe - November/December 2015 - 22
IBM Systems Magazine, Mainframe - November/December 2015 - 23
IBM Systems Magazine, Mainframe - November/December 2015 - Feature: Personal Shopper: Retailer optimizes analytics to drive a more customized client experience
IBM Systems Magazine, Mainframe - November/December 2015 - 25
IBM Systems Magazine, Mainframe - November/December 2015 - 26
IBM Systems Magazine, Mainframe - November/December 2015 - 27
IBM Systems Magazine, Mainframe - November/December 2015 - 28
IBM Systems Magazine, Mainframe - November/December 2015 - Tech Showcase: In a roundtable, distributors discuss how IBM LinuxONE enables flexibility for users
IBM Systems Magazine, Mainframe - November/December 2015 - 30
IBM Systems Magazine, Mainframe - November/December 2015 - 31
IBM Systems Magazine, Mainframe - November/December 2015 - 32
IBM Systems Magazine, Mainframe - November/December 2015 - 33
IBM Systems Magazine, Mainframe - November/December 2015 - Tech Corner: Runtime Diagnostics and Predictive Failure Analysis V2.2 improve z/OS availability
IBM Systems Magazine, Mainframe - November/December 2015 - 35
IBM Systems Magazine, Mainframe - November/December 2015 - 36
IBM Systems Magazine, Mainframe - November/December 2015 - 37
IBM Systems Magazine, Mainframe - November/December 2015 - Solutions: Strobe 5.2
IBM Systems Magazine, Mainframe - November/December 2015 - 39
IBM Systems Magazine, Mainframe - November/December 2015 - Stop Run: IBMer volunteers time to educate people about malaria and technology
IBM Systems Magazine, Mainframe - November/December 2015 - Cover3
IBM Systems Magazine, Mainframe - November/December 2015 - Cover4
IBM Systems Magazine, Mainframe - November/December 2015 - RF1
IBM Systems Magazine, Mainframe - November/December 2015 - SE1
IBM Systems Magazine, Mainframe - November/December 2015 - SE2
IBM Systems Magazine, Mainframe - November/December 2015 - SE3
IBM Systems Magazine, Mainframe - November/December 2015 - SE4
IBM Systems Magazine, Mainframe - November/December 2015 - SE5
http://www.ibmsystemsmagmainframedigital.com/mspcomm/ibmsystemsmag/ibmsystems_mainframe_20200910
http://www.ibmsystemsmagmainframedigital.com/mspcomm/ibmsystemsmag/ibmsystems_mainframe_20200708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20200506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20200304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20200102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/2020mfse
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20191112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20190910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20190708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20190506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20190304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/relevantz_20190102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/2019mfse
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20190102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20181112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20180910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20180708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20180506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20180304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20180102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/2018mfse
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20171112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170910_v2
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_sesupp
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20170102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_linuxsupp
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20161112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/MainframeSecurity
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20160910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20160708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20160506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20160304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20160102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20151112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150910_se
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150910
http://www.ibmsystemsmagmainframedigital.com/MFSkills
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150506_supp
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20150102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20141112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140910_v2
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_gt_201405
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/BigData
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20140102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20131112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130910_v2
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20130102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20121112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/buyersguide2013
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20120910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20120708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20120506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20120304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20120102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/ibmsystems_mainframe_2012bg
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20111112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20110910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20110708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20110506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20110304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20110102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20101112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100910_bg
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100506
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100304
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20100102
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20091112
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20090910
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20090708
http://www.ibmsystemsmagmainframedigital.com/nxtbooks/ibmsystemsmag/mainframe_20090506
https://www.nxtbook.com/nxtbooks/ibmsystemsmag/mainframe_20090304
https://www.nxtbookmedia.com