This post introduces you to emerging AI technologies that will potentially lead growth of artificial intelligence applications for next two to five years.

1. Generative Adversarial Networks

Generative Adversarial Networks (GANs) use two models: a generator and a discriminator, both trained with same data. Generator generates new examples from given data and throws them to discriminator, along with some original or real examples. Discriminator classifies these samples as real or fake. Two models work as adversaries. With each round, discriminator gets better at telling real from fake as generator at creating better fakes. Read more about GANs here.

2. Capsule Networks

When our eyes see a 3D object, it can identify hierarchical relationships between object parts. In simple terms, humans can identify objects from different poses! But internal data representation of a convolutional neural network does not take into account important spatial hierarchies between simple and complex objects. This is where capsule networks can help.

Capsule theory has two important parts : collection of neurons called “capsules” and an algorithm for “dynamic routing between capsules”. The algorithm allows capsules to communicate to create what will be similar to scene graphs in computer graphics. This can drastically improve the efficiency of image classification or object identification tasks. Further reading on Capsule Networks.

3. Conscious Machines

Machine consciousness means that the machine is aware of situation or fact. In scary terms, it is like “Skynet of Terminator series becoming self-aware.” In demonstration of this, Columbia Engineering researchers have created a robot that learns what it is. Without any prior knowledge of its build, the robot can create a self-simulation. The robot can then use that self-simulator internally to contemplate and adapt to different situations, handling new tasks as well as detecting and repairing damage in its own body. Further reading on conscious machines : Forbes, Columbia University

4. Contextual AI

Contextual AI refers to applications that can understand user’s context. The system can see the human perspective with enough information about the environment, situation and context. Contextual AI makes applications more personalized. For example, a smart home assistant knows your preferences and learns your habits to provide a more personal experience. Read more about contextual AI at IBM

5. Custom AI Chips

There was a time when only gamers needed GPUs where as today GPUs are used in variety of ML and AI applications.Given the rise of GPU, chip manufacturers such as Intel are creating specialized chips with computing power up to 3 TOPS (Trillion operations per second). Few examples are : Intel Neural Compute Stick

6. Debating Systems

Think of a system that scans newspaper and magazine articles to present a dueling narrative for a topic. That’s what IBM’s Project Debater did! Such systems can help humans build persuasive arguments and make better informed decisions.

What other emerging AI technologies to include in the list? Put your suggestions in the comments below.

In this tutorial I’ll show you building a movie recommendation service with Apache Spark. Two users are alike if they rated a product similarly. For example, if Alice rated a book 3/5 and Bob also rated the same book 3.3/5 they are very much alike. Now if Bob buys another book and rates it 4/5 we should suggest that book to Alice, that’s what a recommender system does. See references if you want to know more about how recommender systems work. We are going to use Alternating Least Squares method from MLLib, and MovieLens 100K dataset which is only 5 MB in size. Download the dataset from https://grouplens.org/datasets/movielens/. Code :

from pyspark.mllib.recommendation import ALS,MatrixFactorizationModel, Rating
from pyspark import SparkContext

sc = SparkContext ()

#Replace filepath with appropriate data
movielens = sc.textFile(“filepath/u.data”)

movielens.first() #u’196\t242\t3\t881250949’
movielens.count() #100000

#Clean up the data by splitting it,
#movielens readme says the data is split by tabs and
#is user product rating timestamp
clean_data = movielens.map(lambda x:x.split(‘\t’))

#We’ll need to map the movielens data to a Ratings object
#A Ratings object is made up of (user, item, rating)
mls = movielens.map(lambda l: l.split(‘\t’))
ratings = mls.map(lambda x: Rating(int(x[0]),\
int(x[1]), float(x[2])))

#Setting up the parameters for ALS
rank = 5 # Latent Factors to be made
numIterations = 10 # Times to repeat process

#Need a training and test set, test set is not used in this example.
train, test = ratings.randomSplit([0.7,0.3],7856)

#Create the model on the training data
model = ALS.train(train, rank, numIterations)

For Product X, Find N Users to Sell To

model.recommendUsers(242,100)

For User Y Find N Products to Promote

model.recommendProducts(196,10)

#Predict Single Product for Single User
model.predict(196, 242)

References:

1. Building a Recommender System in Spark with ALS, LearnByMarketing.com
2. MovieLens
3. Video : Collaborative Filtering, Stanford University
4. Matrix Factorisation and Dimensionality Reduction, Thierry Silbermann
5. Building a Recommendation Engine with Spark, Nick Pentreath, Packt

https://twitter.com/alexstamos/status/916697104197627904 This recent tweet by Facebook’s security chief Alex Stamos stimulated my thinking about literature reading of Silicon Valley executives and importance of humanities education to the tech students. Stamos argues here that his company should not become “Ministry of Truth” (the propaganda machine in Orwell’s 1984) by running algorithms on deciding what is truth and what is propaganda or fake news. An analogy from Orwellian classic helped Stamos explain a complex evil scenario in simple terms. But is the role of cultural literacy limited to simplifying otherwise complex scenarios? No, if culture is defined by the values of its practitioners, I think it cannot be separated from anything they create. We create Artificial Intelligence. An algorithm cannot be neutral when it is designed by biased humans. An undiscriminating AI model trained by discriminating expert is improbable. We need to find ways to pass our values to what we are set to create. Fortunately, tech leaders do understand this. Emma Williams is a general manager at Microsoft whose area is Anglo-Saxon literature. Her role is to ensure that Cortana’s personality is calm and sober. Cortana is Microsoft’s premiere chat AI. It is evident that identifying role of human values in technology has never been more important than now. In Cortana’s case, for example, who would want to chat with a short tempered AI? [caption id=”attachment_202” align=”alignnone” width=”436”] Satya Nadella Quotes Ghalib at a Presentation in New Delhi[/caption] “Hazaaron khwaishein aisi, ke har khwaish pe dum nikle. Bohat nikle mere armaan, fir bhi kam nikle,” people were surprised when Microsoft CEO Satya Nadella quoted great Rekhta poet Ghalib during a presentation in New Delhi last year. ‘Yet another geek into poetry’ one may wonder. But it perfectly suites CEO of a tech giant whose products aspire to be as good as human in some ways. If poetry can ease complexity of affairs by creatively deploying words why shouldn’t robots use them? Also, ill-informed reading in history undermines innovations and experiments in social and political thoughts. Which I think is a great cause of concern for tech students. For example, civil disobedience as an idea may not occupy any place in the mind of an engineer as a revolutionary approach towards fighting oppression. It is only hypocrisy talking about singularity and at the same time denying human culture and values any place in there. If future is what we build today, we should build it good and not evil. In India universities are finally moving towards offering more diverse learning experience to the students where an engineering student can study Shakespeare. Though the progress is very slow the outcomes should be positive with CBCS or choice based credit system. References:

1. Facebook Stumbles With Early Effort to Stamp Out Fake News, Sarah Frier, Bloomberg
2. The Bing Search Experience: A Practice In Italian Renaissance Art & User Perspective, Amy Gesenhues, Search Engine Land
3. Yes, Microsoft CEO Satya Nadella can quote Mirza Ghalib, The Indian Express
4. Choice based credit system: the path ahea, M.S. Bhat, The Hindu