# What is geometric deep learning?

What is geometric deep learning (GDL)?How is it different from deep learning?Why do we need GDL?What are some applications of GDL?

• I have the impression it's gonna be closed as too broad soon,however I appreciate the impulse it's giving to explore this new field Nicola Bernini Mar 19 at 9:17
• @NicolaBernini You don't have to describe in detail all aspects of GDL.I 必威电竞asked a few sub-必威英雄联盟questions,so that people can focus on those.I 必威电竞asked other more specific but related 必威英雄联盟questions:www.1kvaups.com/必威英雄联盟questions/tagged/geometric-deep-learning.I will eventually improve my answer,as soon as I have a more solid knowledge of the field (and some more time). nbro Mar 19 at 10:03

To complete thefirst answerthat is rather graph oriented,I will write a little about deep learning on manifolds,which is quite general in terms of GDL thanks to the nature of manifolds.

Note that the description of GDL through the explanation of what are DL on graphs and manifolds,in opposition to DL on euclidean domains,comes from the 2017 paperGeometric deep learning: going beyond Euclidean data(this paper is excellent at clarifying both the intuition and the mathematics of what I'm writing).

## 1.In case you don't know what a manifold is

As the previously cited paper puts it:

Roughly,a manifold is a space that is locally Euclidean.One of the simplest examples is a spherical surface modeling our planet: around a point,it seems to be planar,which has led generations of people to believe in the flatness of the Earth.Formally speaking,a (differentiable) d-dimensional manifold X is a topological space where each point x has a neighborhood that is topologically equivalent (homeomorphic) to a d-dimensional Euclidean space,called the tangent space.

Good other not-so-technicalexplanationon stats.stackexchange

OtherWikipedia examplesto develop not too abstract understanding

Very shortly put,it's an interesting mathematical set on which to work (different kinds exist,see papers at the end of this answer for DL related manifolds uses).By work,you can typically understand that youconstrain the neural net parameters to the manifoldyou chose (e.g.training with parameters constrained on a hypersphere,among thegeomstats paperexamples).

Your data can also be represented thanks to a practical manifold.For example,you can choose to work on images and videos by representing the samples using Symmetric Positive Definite (SPD) matrices (see thispaper),the space of SPD matrices being a manifold itself.

## 2.Why bother learning on manifolds ?

Defining a clearer/better adapted set (understand that it's a sort of constraint!) on which to learn parameters and features can make it simpler to formally understand what your model is doing,and can lead to better results.I see it as a part of the effort of deep learning formalization.One could sayyou're looking for the best information geometry for your t必威电竞ask,the one that best captures the desirable data distribution properties.To develop this intuition consider thesolar system analogy for manifold learning of this Kaggle kernel:

Perhaps a good analogy here is that of a solar system: the surface of our planets are the manifolds we're interested in,one for each digit.Now say you're on the surface of the earth which is a 2-manifold and you start moving in a random direction (let's assume gravity doesn't exist and you can go through solid objects).If you don't understand the structure of earth you'll quickly find yourself in space or inside the earth.But if you instead move within the local earth (say spherical) coordinates you will stay on the surface and get to see all the cool stuff.

This analogy reminds us of the spherical surface planet model fromBronstein's paperalready quoted above.This paper also describes a typical case for which manifolds are interesting: where graphs (the other example of GDL/DL on non euclidean data) are better at handling data from social or sensor networks,manifolds are good at modeling 3D objects endowed with properties like color texture in computer vision.

## 3.Regarding deep neural networks on manifolds

I would advise reading thegeomstats associated paper,which does a great job at showing what it is and how it can be used,along with example codes (e.g.MNIST on hyperspheres manifold example codehere).This library implements manifolds and associated metrics on Keras.The choice of metrics is essential to understand the point of working on manifolds: it's because you need to work on an adapted mathematical set (iewith the right properties) with an adapted distance definition (so that the measure actually means something when considering the problem you're trying to solve) that you switch to working on manifolds.

If you want to dive in the details and examples of deep learning on manifolds here are some papers:

## 4.WhyRiemannian集合管吗?

TL;DR: you need a metric to do machine learning(otherwise,how could you evaluate how much you actually learned !)

Still based onBronstein's paper:

On each tangent space,we define an inner product [...].This inner product is called a Riemannian metric in differential geometry and allows performing local measurements of angles,distances,and volumes.A manifold equipped with a metric is called a Riemannian manifold.

## 5.What's the relation between a Riemannian manifold and a Euclidean space ?

Still based onBronstein's paper:

a Riemannian manifold can be realized as a subset of a Euclidean space (in which case it is said to beembeddedin that space) by using the structure of the Euclidean space to induce a Riemannian metric.

I leave the details to the paper,otherwise this answer will never end.

Will only answer once I think I've found a relatively well-argued answer,so won't answer everything at once.

• Isn't manifold learning just a way of dimensionality reduction?

I don't think so,it isn'tjust that.I haven't seen any dimensional reduction constraint (yet ?) in the papers I've read (cf.geomstats again).

In thehypersphere/MNIST geomstats code example,you can see the chosen manifold dimensionhypersphere_dimension = 17.Since we're working with MNIST data I guess this would mean a dimension reduction in this particular case.I admit I would need to check exactly what that dimension implies on the neural net architecture,I haven't discussed my understanding of this yet.

## Disclaimer

I'm still developing a more rigorous mathematical understanding of manifolds,and shall update this post to make additional necessary clarifications: exactly what can be considered as a manifold in a traditional deep learning context,why do we use the word manifold when speaking about the hidden state of auto-encoders (see the previously cited Kaggle kernel that quotes Goodfellow's book on this).All of this if the perfectly clear answer doesn't show up here before !

• I didn't talk about manifolds because I am not very familiar with this concept,and I think that many CS/AI students will not be familiar with such concept.I think you should give an intuitive and brief explanation of the meaning of a manifold,in order to fill this gap.How do you intuitively think of a manifold?Isn't a manifold also "Euclidean data"?Why isn't e.g.an image a manifold?I think your answer would improve a lot if you explain and clarify these points.Btw,see this related question:www.1kvaups.com/q/11226/2444.Feel free to answer it,if you are able to do it. nbro Mar 15 at 10:21
• I think you should also consider the fact that people are used to look at datasets as a design matrix or a bunch of feature vectors (for each observation).How would this compare to manifolds and non-Euclidean data?Maybe,if you are able,you should answer these 必威英雄联盟questions on the linked and related question. nbro Mar 15 at 10:28
• Isn't manifold learning just a way of dimensionality reduction? nbro Mar 15 at 13:43
• Manifold is easy to understand if your understand euclidean space.Euclidean space is a space of n-dimensional vectors of real numbers.Manifold is a space (set of) vectors that for any point of it small area around it could be transformed (smoothly and reversibly) to ball of Euclidean space.So manifold is locally almost Euclidean,but globally generally is not.That mean that every small area of manifold can be approximated by hyperplane (the smaller area,the more precise),but globally it can not be.Manifolds are introduced in differential topology and differential geometry. mirror2image Mar 18 at 9:05
• @mirror2image You should have described what an Euclidean space is.What are the necessary properties that a space must have to be considered Euclidean?You should also explain what you mean by "locally" and "globally". nbro Mar 18 at 20:48

The articleGeometric deep learning: going beyond Euclidean data(by Michael M.Bronstein,Joan Bruna,Yann LeCun,Arthur Szlam,Pierre Vandergheynst) provides an overview of this relatively new sub-field of deep learning.It answers all the 必威英雄联盟ques必威电竞tions asked above (and more).If you are familiar with deep learning,graphs,linear algebra and calculus,you should be able to follow this article.

What is geometric deep learning (GDL)吗?

Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds.

So,the inputs to these GDL models are graphs (or representations of graphs),or,in general,anynon-Euclidean data.To be more concrete,the input to these models (e.g.graph neural networks) are e.g.feature vectors associated with the nodes of the graphs and matrices which describe the graph structure (e.g.theadjacency matrixof the graphs).

Why are e.g.graphs non-Euclidean data?

A graph is a non-Euclidean structure because e.g.distances between nodes are not well defined.Yes,you can have graphs with weights associated with the edges,but not all graphs have this property.

What classes of problems does GDL address?

In GDL,there are two classes of problems that are often tackled:

1. characterise the structure of the data (e.g.of a graph)
2. analyse functions defined on a given non-Euclidean domain

These classes of problems are related,given that the structure of the graph imposes certain properties on the functions that can be defined on it.Furthermore,these properties of these functions can also convey information about the structure of the graph.

What are applications of GDL?

An example of an application where this type of data (graphs) arises is in the context of social networks,where each user can be associated with a vertex of the social graph and the characteristics (or features) of each user (e.g.number of friends) can be represented as a feature vector (which can then be associated with the corresponding vertex of a graph).In this context,the goal might e.g.be to determine different groups of users in the social network (i.e.clustering).

Why can't we simply use deep learning methods (like CNNs) when the data is non-Euclidean?

There are several problems that arise when dealing with non-Euclidean data.For example,operations like convolution are not (usually) defined on non-Euclidean data.More concretely,the relative position of nodes is not defined on graphs (but this would be required to perform the usual convolution operation): in other words,it is meaningless to talk about a vertex which is e.g.on the left of another vertex.In practice,it means that we can't simply use the usual CNN when we are given non-Euclidean data.There have been attempts to generalise the convolution operation to graphs (or to approximate it).The field is still quite new,so there will certainly be new developments and breakthroughs.

• I need to improve this answer by explaining more in detail the meaning of "characterize the structure of the data" and "analyze functions defined on a given non-Euclidean domain".I should also explain what the output of these GDLs models are and how can they be trained. nbro Mar 14 at 9:36