IBM Systems Magazine, Mainframe - March/April 2017 - 23
the historical and real-time data. In
order to react intelligently in real
time, we use machine learning on
the historical data to understand
the expected traffic behavior for
each city location and time period.
That way, we know how to react
based on context, such as location
in the city (busy city intersection or
suburban outskirts) and whether
it's a rush-hour period or not.
ISM: How will this help the
PTS: We're helping them become
more efficient in the control rooms
where they currently employ a
lot of people looking at traffic
screens and manually managing
traffic. We cluster the traffic into
so-called good and bad traffic.
This way, when traffic in a certain
location moves from good to bad,
we can raise an alert that might
trigger notification of the Madrid
bus company, alert passengers on
highways via information panels
or call for emergency vehicles.
Our work can help those in the
control rooms react faster. The possibility of automating the response
with little or no human intervention also exists. For example,
one could imagine an automatic
system for adjusting traffic-light
behavior according to the current
We have new speed and intensity
readings [showing the amount of
vehicles] coming in around every
five minutes. If we're looking just
at the city of Madrid, the rate of
data coming in is moderate. But
you can imagine this extended to
many other cities or areas around
It's important that the machine
learning for different locations and
time periods is done ahead of time.
When new data comes in, we've
already calculated thresholds that
tell us if we're switching from good
traffic to bad traffic. And as that
comes in, the machine-learning
"IoT is leading
the way to
new types of
-Paula Ta-Shma, research staff member,
models may need to change over time, so we have a
technique that tells us when our clustering gets out of
whack and we need to run it again.
We're also using a dashboard that monitors traffic
at different locations in Madrid with the capability
to display alerts in locations where the speed or the
intensity of traffic goes beyond a certain threshold.
That threshold is calculated using machine learning.
It's different for each location and for rush-hour periods versus nighttime periods, and so on.
ISM: How do OpenStack Swift and Apache
Spark come into play?
PTS: OpenStack Swift is a scalable, low-cost way
to store data. This is crucial for IoT data, which is
quickly becoming the "biggest" big data ever. That's
because so many things are connecting to the internet
and generating data. We're also making it more
efficient to analyze data on OpenStack Swift for our
Apache Spark is a cluster-computing framework
that enables analytics to be done on the data. So,
OpenStack Swift provides the storage, and Apache
Spark provides the compute power for the analytics.
It's doing the machine learning and computation on
ISM: So, you combined Swift and Spark to
work almost as a single entity?
PTS: We're also using other open-source tools we
helped play nicely together. Apache Kafka and Pinterest Secor are used as a message hub and a way to get
the data into Swift. Apache Parquet, a special-data
format, is good for analytics in general and for IoT
data in particular. We use Elasticsearch to make the
analytics more efficient, and IBM Node-RED hooks
together systems and sensors to improve the management of the IoT.
It's a complex challenge. Clients
don't just want to run the solution
on their laptops. They want it
to be deployed in the cloud and
scalable so they can start with a
small amount of data or number
of users and grow elastically. One
of the exciting outcomes of our
work is our collaboration with
another IBM team, led by Naeem
Altaf, to port the whole Madrid
traffic use case to run on the IBM
ISM: How can this technology
PTS: In COSMOS, for example,
we did some work on occupancy
detection and the ability to save
energy by turning appliances off
when people have left their home
or office. You can potentially do
similar work with other types of
sensors collecting data.
For example, by putting sensors
along pipes and performing
anomaly detection, you can detect
and send a warning if a pipe leaks.
You can even build actuators into
your solutions so that when a pipe
is leaking, the mains automatically
shut off to prevent further damage
to your home. If a company has
an oil pipeline traveling across
the country, you could use similar
technology to help avoid an
ISM: Where do you see the
IoT heading in the future?
PTS: IoT is leading the way to
new types of data being collected
in unprecedented quantities.
For example, in healthcare, you
may use this technology to get
more insight into the reasons
For traffic, we could gain new
insight into the conditions and
locations under which accidents
occur. Having this additional data
can provide new opportunities and
approaches we may never have
ibmsystemsmag.com MARCH/APRIL 2017 // 23
pg 22-23.indd 3
2/7/17 1:27 PM