Deep Learning for Advanced Analytics

This is my third post on tensors and tensor decompositions. The first post on this topic primarily introduced tensors and some related terminology. The second post was meant to illustrate a particularly simple tensor decomposition method, called the CP decomposition. In this post, I will describe another tensor decomposition method, known as the Tucker decomposition. While the CP decomposition’s chief merit is its simplicity, it is limited in its approximation capability and it requires the same number of components in each mode. The Tucker decomposition, on the other hand, is extremely efficient in terms of approximation and allows different number of components in different modes. Before going any further, lets look at factor matrices and n-mode product of a tensor and a matrix. Factor Matrices Recall the CP decomposition of an order three tensor expressed as X≈∑r=1Rar∘br∘cr, where (∘ ) represents the outer product. We can also represent this decomposition in terms of organizing the vectors, ar,br,cr,r=1,⋯R , into three matrices, A, B, and C, as A=[a1a2⋯aR], B=[b1b2⋯bR],and C=[c1c2⋯cR] The CP decomposition is then expressed as X≈[Λ;A,B,C], where Λ is a super-diagonal tensor with all zero entries except the diagonal elements. The matrices A, B, and C are called the factor matrices. Next, lets try to understand the n-mode product. Multiplying a Tensor and a Matrix How do you multiply a tensor and a matrix? The answer is via n-mode product. The n-mode product of a tensor X∈RI1×I2×⋯IN with a matrix U∈RJ1×In is a tensor of size I1×I2×⋯In−1×J×In+1×⋯×IN, and is denoted by X×nU . The product is calculated by multiplying each mode-n fibre by the U matrix. Lets look at an example to better understand the n-mode product. Lets consider a 2x2x3 tensor whose frontal slices are:

In Part 1 of this blog, I had explained what tensors are and some related terminology. Just to briefly recap, a tensor is a multidimensional array of numbers. Tensors provide a useful representation for data that involves multiple moda

This is a three-part blog post about tensors. The first part introduces tensors; the second and third part discuss tensor decomposition. So a natural question that you might have is what is a tensor and why should I be interested in them? Well read on to get your answer.Simply stated, a tensor is a multidimensional or N-way array. The array dimensionality, the

In an earlier post, I had explained convolution and deconvolution in deep neural networks. The purpose of this post is to demo these operations using PyTorch. Before doing that, we will visit different operations associated with a convolution operation.

One of the convolution layer’s parameters in PyTorch is the groups parameter. This parameter controls the connections between the input and output channels. In this post, I will describe different scenarios for the group parameter to better understand it.

There are neural network applications where the resources, for example compute power, memory space, battery power etc., are limited. In such instances, we may want to simplify a trained network to better deal with available resources. While simplifying, we obviously would like to maintain the network accuracy as much as possi

This is second part of a three part series of blog posts about compression of deep neural networks. The compression, which reduces the number of parameters and computation, is a useful technique for deploying neural ne

This is the final post of a three part series on compressing convolutional neural networks. As indicated in the previous blogs of this series, sometimes you have a resource crunch and you need to reduce the number of parameters of a trained neural network via compression to work with available resources. In my earlier blog post on this topic, I had shown how to reduce the number of connection weights of a fully connected layer of a convolutional neural n

Recently, I came across an article, Denoised Smoothing: A Provable Defense for Pretrained Classifiers, that suggested a method to deal with adversarial examples. The method consists of creating multiple noisy copies of an input image; these are then filtered using a custom-trained denoiser a

Have you often wondered about the difference between the semi-supervised learning and the self-supervised learning? Well! This post might be of help. Let us begin with supervised learning, the most popular machine learning methodology to build predictive models. It uses annotated or labeled data to train a predictive model. A label attached to a data vector is nothing but the response that we expect from a predictive model with that data vector as the input during the model training. For example, we will label pictures of cats and dogs with labels cat and dog while building a Cat versus Dog classifier. When building a classifier, we assume a large enough training data set with labels is available.

Building a classification model with a collection of annotated training examples requires making the following choices: