Blockchain apps must be closed systems

by Srikumar Subramanian

Blockchain tech, especially smart contracts, are the hot new “internet”. Post the creation of Bitcoin, we’ve seen the rise of the public smart contract system Ethereum and several “private” systems like The Linux Foundation’s Hyperledger. These distributed ledgers have become the hot new foundation to build apps on top of, leveraging the additional trust that they are supposed to provide by virtue of their distributed nature.

Interactive Fluid Simulation

by Ramakrishnan Mohan

Realistic physics-based simulations are a great way to engage a user in a gestural interactive system. Nowadays we witness such systems in shopping malls that allow us to try on clothes virtually, or play games. Among such simulations, it has been found that interacting with fluids such as water has a calming effect. We wanted to replicate this feeling digitally as an interactive wall. Capturing how water moves using a finite element simulation of a water surface turned out to be an interesting challenge in our exploration.

In this post, we share some interesting results from this experiment in which we implement a fluid simulation system towards such a water surface that a user can play with, as well as a few alternative, colourful visualizations of disturbed viscous fluids. The interactions were effected using a Kinect2 and the visualizations were projected on a wall for an immersive effect as shown in the following videos.

Deep Type

by Manoj Kumar et al
At Imaginea, we run a social network for typoholics called Fontli as our designers have a passion for the field. Folks share typography that they catch in the wild or work that they’ve created themselves. Members ask others for font identification and tips, and tag what they’re able to identify themselves. Given that we’re into typography, we would love to have a system where we can take a picture of some type and apply it to text of our own choice!

Typography Detection

by Uttam Erukala et al

Keeping undesirable content out of social networks and communication channels is a common problem. Our email systems today have sophisticated “spam filters” thanks to which we’re protected from much harm and waste of time. The problem of spam is particularly harsh in niche social networks and interest groups which are small and sensitive to disruption. We run one such niche social network for typography enthusiasts called Fontli and we like to protect our dear typographers from content that they’re not interested in - which is everything that isn’t typography. The problem is that this is hard … even for humans!

In this post, we talk about a filter we recently developed and deployed to reduce and flag incidences of non-typographic content on Fontli, using a deep convolutional neural network based image classifier. We’ve had modest success and faced some intriguing situations and results along the way.

Hello world

by Labs et al

Please welcome Imaginea Labs to the world. This is our new online home now where we’ll be sharing our trials and tribulations with the tech that we grapple with.

Spark 2.0 SQL source code tour part 1 : Introduction and Catalyst query parser

by Bipul Kumar

PS: Cross posted from blog.imaginea.com

While digging through Apache Spark SQL’s source code, what I realized is that looking deeper into its source code is a must for all those who are interested in implementation details of any SQL based data processing engine. Spark SQL has three major API modules: Datasource Api, Catalyst Api and Catalog Api. This series of posts concentrate mostly on the Catalyst correlating the query processing stages(Parsing, Binding, Validations, Optimization, Code generation, Query execution) to highlight all the entry points for further exploration.

PREREQUISITE: Overview of Spark Catalyst. You can start with this link.

NOTE: Code snippets in these posts refer to Spark SQL 2.0.1.

As we know since 2.0 SparkSession is the new entry point for the DataFrame through Dataset, is associated with a SessionState which wraps the SQLContext.

Investigations on Spark Join

by Sachin Tyagi

(Note - this post is extracted from an email on the topic and has been posted on labs.imaginea.com retroactively.)

TLDR: There are things you can do to improve your RDD join performance on Spark in some cases.