Impressions and Links from
MLCon 2021

The Conference for Machine Learning Innovation
December 7 - 8, Berlin.

Equestrian statue of Frederick the Great, Berlin

I had the great pleasure of taking part in MLCon 2021 (The Conference for Machine Learning Innovation. December 7 - 8, 2021. Berlin).

Below you will find impressions from the conference, and links for further reading.

The MLCon 2021 conference was held at the Maritim proArte Hotel Berlin in Berlin, Germany.
Not that far away from Unter den Linden, and the Humboldt-Universität.

Tried to follow as many talks as possible. But, well, these notes are, of course, in no way, shape or form complete...
Rather, these notes were written on conference nights, as my way of keeping track of the events that I attended at the conference. And as a way of storing links and references for future reference.

1. Impressions from Tuesday, December 7th.

1.1. Teaching AI common sense.

Pieter Buteneers talked about ''AI and common sense''.

Clearly, humans and machine learning algorithms do not learn about the world in the same way.

- Humans can learn from just a few examples, while machine learning algorithms can not.

In supervised learning the machine learning algorithms need a lot of labelled data, in order to e.g. learn whether a picture contains a cat, or if the data is good or bad, high or low.
Problematic as labelled data is costly.

And there do seem to be a long way to go before algorithms are anywhere near human common sense...

Or, well maybe not so fast, sometimes ML algorithms, trained using self-supervised learning,
do seem to pick up common sense, a little bit a least.

In this interesting talk we got a short introduction to BERT:

BERT is designed to help computers understand the meaning of ambiguous language in text by using surrounding text to establish context [1].

and other, more advanced, language models, that begin to show, at least, some early signs of having learned some ''common sense'' during training. With more advanced models becoming available on the internet on a daily basis.
See e.g. here: GPT Neo & Huggingface models.

Indeed, an interesting talk, and a good start to the conference
(For more about ''AI & common sense'', see my post here).

Need more solid proof, i.e. running software, and not completely convinced just yet?

Well, then you might find Mat Velloso's comment appropriate here...

1.2. Advances in Machine Learning Systems.

Andreas Bühlmeier talked about ''Advances in Machine Learning Systems
– from GPUs to quantum computers''.

I.e. ''Many achievements of Machine Learning have been possible only by using specific new hardware or distributed approaches''.
Here in this talk with focus on:

Neuromorphic Engineering.
GPU accelerated ML.
Cluster Machine Learning, with Apache Spark.
Quantum Computing.

Especially, the presentation of the Qiskit, Open-Source Quantum Development, was interesting, and, probably, new to most of the audience (here, december 2021).

About Qiskit:

Qiskit accelerates the development of quantum applications by providing the complete set of tools needed for interacting with quantum systems and simulators.

1.3. Keep an eye on AI — Monitor ML models in production.

Hauke Brammer, Finpair GmbH, talked about ''Monitoring ML models in production''.

Well, monitoring a ML-system is hard.
You have code, model and data. And all of it is intertwined.

Sure, code might work just as well after a month, but data might get outdated, which will give us ''input drift'', followed by ''prediction drift''.

So, we need to monitor the deployment, and go back and retrain the model, if necessary.

Here it was suggested to use Prometheus to monitor the solution.

Use Elastic to:

Find anomalies and outliers, forecast based on trends, and identify areas of interest in your data with Elastic machine learning.

And use Pachyderm to:

Continuously update data in the master branch of your repo, while experimenting with specific data commits in a separate branch or branches.

Which, all in all, gave a lot of ideas on:

Monitoring machine learning models in production. Making sure that the model still delivers business value
- by detecting failure modes of the ML models before they turn into big disasters.

All, super useful stuff indeed.

1.4. Affective Computing – bringing humans and machines closer through emotions?

Håkan Silfvernagel, Miles, talked about ''bringing humans and machines closer through emotions''?

I.e.

Research has shown that emotions play an integral role in decision making, cognition, perception, learning, and more.
If we want our computers to be intelligent and be able to interact with us we need to ensure that they are able to recognize, understand and express emotions.

In education, people would like to be able to monitor whether the students are bored.
And in advertising, people would like to know what mood people are in, in order to be able to inject proper advertising.
Etc.

So, people want this technology...?
Well, some people...

Will such technologies ever work?

The challenge is that building an algorithm to detect someones emotions ignores cultural differences and other important factors [2].

Scientists have been trying for decades to codify the facial expressions linked to different emotions, and yet it’s never worked [2].

Well, if only we had smarter computers then.
Like, (future) computers that could pass:

The coffee test (be able to make coffee) [3].

The robot college test (go to college for us).

The employment test (do our, office, jobs for us).

The Turing Test (talk about everything, like a person).

Then surely these computers would also be (should be?) able to decode facial expressions?

Well, perhaps.

The whole thing is (still) pretty controversial.
According to Kate Crawford:

There is no good evidence that facial expressions reveal a persons feelings.
But big tech companies want you to believe otherwise [4].

According to Crawford, these technologies are not just technically imperfect, they are also based on questionable methodologies.
Still:

Job applicants are judged unfairly because their facial expressions or vocal tones don’t match those of other employees. Students are flagged at school because their faces appear angry, and customers are questioned because their facial cues indicate they may be shoplifters.

Indeed, emotions are not just bringing us together, they are also tearing us apart.

1.5. Efficient Transformers.

Christoph Henkelmann, Divisio talked about transformers:

I.e.

Transformers are the new go-to technology for Natural Language Processing (NLP).
But, despite all their successes and widespread adoption, they have one major drawback:
Their computation and memory requirements grow quadratically with the input size.
So, training transformer models from scratch is a very resource-intensive task.

Here, in this talk we were then given a tour of recent research into efficient transformer layers, that works with smaller computation and/or memory requirements.

The illustrated transformer:

Here, Christoph Henkelmann pointed us in the direction of Jay Alammar.

Where Jay Alammar explains what a transformer is, here.

See also the paper:
''Attention Is All You Need''
(the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely).

(Among many other interesting things) on Jay Alammars homepage one can find a notebook for using BERT (a transformer-based machine learning technique for natural language processing), as a beginner, here.

Henkelmann then pointed us in the direction of mT5 (A massively multilingual pre-trained text-to-text transformer).

And ''Big Bird, transformers for Longer Sequences'' (Using a sparse attention mechanism that reduces BERTs quadratic dependency to linear).

Indeed, all in all, a good overview, and a brilliant talk.

1.6. Natural Language Processing (NLP): The Transformer Zoo.

Christian Winkler, Datanizing GmbH, & Oliver Zeigermann, Embarc, talked about ''The Transformer Zoo''.

''Transformer-based'' language models have been at the core of the recent progress in Natural Language Processing (NLP).
Starting with the original BERT, we here take a look at derivatives like RoBERTa and ALBERT, stressing the differences to systems like GPT-2 & GPT-3.
Finally, we show how the myriad of (fine-tuned) models available at Huggingface
can be used very easily in Jupyter notebooks to solve real-world problems in different industries.

On Huggingface there are thousands of pretrained (transformer) models to perform tasks on different modalities such as text, vision, and audio.

These models can be applied on:

Text, for tasks like text classification, information extraction, question answering, summarization, translation, text generation, in over 100 languages [5].

Tutorial, getting started with models from Huggingface:

Using libraries from the Hugging Face ecosystem. Transformers, Datasets, Tokenizers, and Accelerate. See here.

Things like sentiment detection is pretty straight forward with models from Huggingface.
See e.g. T5 (finetuned emotion), Roberta (large english) & Twitter Roberta.

And much, much more is, of course, possible with models like GPT-3.
Where one can try out the GPT-3 transformer on OpenAi.

Explore the OpenAI API, here.

OpenAI Codex can translate natural language to code...

Transformer Zoo. ML. Berlin. December 2021

Awesome stuff, indeed!

Where one can read more in Christian Winklers book, ''Blueprints for Text Analytics Using Python: Machine Learning-Based Solutions for Common Real World (NLP) Applications'' [6].

2. Impressions from Wednesday, December 8th.

2.1. How to become a fully conversational company?

Jim Tonge, VUI.agency talked about ''How to become a fully conversational company''.

Through conversational user interfaces, machines can now speak, and understand human language.

Voicebots and chatbots are beginning to take service line calls, and answer chat messages - With deep integration into existing business software.
Indeed, whenever a significant volume of employees are doing repetitive communication tasks, or you need to provide 24/7 service 365 days a year, there might be a businesscase for using conversational user interfaces, aka. digital assistants, for automating such customer communication.

According to Tonge: ''Generation voice is here''.

At VUI.agency, we create virtual assistants with charisma. Our uniquely skilled team of 40 top international talents covers everything from Conversational AI strategy, Voice UX, Assistant Persona Design, Conversation Design, and Voice Development.

A conversational interface must show:

Smartness. Make variations, so it doesn't become to repetitive.
Politeness.
''I didn't understand what you said''.
Should be replaced with
''I didn't understand'', where it takes the blame.

And how human-like do we want our chatbots to be?

Should an AI be sleepy in the morning?
Would we trust such an AI?
Should the AI know that it is an AI?
Or is it just something that represents a person?

More conference impressions... E.g. see ...

2.2. Natural language processing for medical data.

Anja Pilz, Damedic, talked about ''natural language processing for medical data''.

It is not unusual that doctors spend more time documenting (70 %) that actually treating someone.

So, it would be very nice, if we had software that could make the documentation process easier and more precise.
And, in the documents, it would certainly be nice, if the software could detect relevant diagnoses, procedures and drugs.
But not easy as medical texts can be super-condensed, and filled with abbreviations.

Luckily, helpful tools might be available in the not so distant future...
And, indeed, an interesting and insightful talk about the problems and solutions within this domain of medical informatics.

**2.3. The ''Ask Me Anything'' session.**

Many good points from the panel.

That also recommended that we took a look at Yannic Kilchers ''ML Youtube'' Channel.
Certainly, videos about Attention is all you need [7] and GPT-3 [8] came accross as ''must see'' videos.

2.4. Using AI to make recommendations for career progression.

Dorra Nouira, Boostrs talked about ''Using AI to make recommendations for career progression''.

With thousands of job titles available, it is difficult for candidates to know what each role entitles and how well-suited they are for various positions.

Our research aims to break though this complexity and identify fitting career progressions for every job.

In the process, we use algorithms that find semantic similarity between job descriptions and skills required for the jobs.
Then we cluster jobs and skills in order to help map-out possible career progressions.

Many career paths are computed based on the similarity between the start and end job.
Where it is often soft skills that can power career transitions.