MLHCI Student projects
                         Competitions

We in the group like challenges, competitions. On this page we have gathered information about various challenges, competitions that are of interest to us. The idea is that a prospective student or a few students start working on a project where the goal is to participare (or win!) the given competition. The MLHCI group provides the tools, experience, knowledge, the "atmosphere", whilst student do the real work, ie. the programming stuff:) Solving challenges is really fun since you know what you are aiming for and that your work can even have some impact! And you can be sure that you are working on cutting edge problems!!!

So here are a the promised links. These point typically to past years' competitions. These competitions are held annually, some are held biannually. On these links you can look up past years results, datasets, participants, evaluation criteria, etc. After the links the very essence of the competition is summarized in a short sentence. Note that many of the underlying datasets are already available locally in the group! If you are interested in one of these competitions then take a look at the previous years solutions, check if you qualify to work with us according to our guide and if the answer is yes, then contact us. If you find some other competition and you think that it would be of interest to us then please let us know that as well! Remember, the goal is to some work while we have lots of FUN!

Planning, control, reinforcement learning

• Probabilistic planning competition; Task is to write a program that can accept the description of a stochastic domain (where things do not always happen as expected) and the description of a task to accomplish (e.g. build a tower). The program should optimize some utility; e.g. maximize the probability of solving the task, minimize expected time to solve the task. Related to reinforcement learning.

• Computer Controlled Car Racing; You have a video feed and your program should control a racing car; must be fun! 

Data mining, knowledge discovery, machine learning

• KDD-Cup; Solve some data mining problem (classification, prediction, etc.) that typically involve difficult datasets (many features, large datasets, specific criteria, etc.)

• NIPS Feature Selection Competition; Given some datasets with a large number of irrelevant features find efficient algorithms that solve some data mining/machine learning tasks.

• (Neural net) time series prediction; actually this year IJCNN is held here in Budapest! You should know what time series prediction is about.

• EUNITE competitions; e.g. Churn reduction, banking industry, Electric load forecasting; Prediction of product quality in glass manufacturing process

Natural Language Processing

• "Learning DFA from Noisy Samples" Given a training set of binary sequences, each with a binary class label, learn a predictor that will classify unlabelled sequences in the test set. The training and test sets consist of random binary sequences labelled by a randomly constructed Deterministic Finite Automata (DFA). This is actually a pretty hard task (as are all the others!)

• Context free grammar modeling The task is to infer a model of a context-free language from unstructured examples (both positive and negative) and to use that model to label a set of test sentences as being either in or out of the language.

• NIST Automatic Content Extraction (similar to previous MUCs). This is an NLP task, write programs that extract the meaning (content) of textual documents.

• Let us also mention here the Loebner Prize Contest that seem to have trouble finding funds. This is basically the Turing test that came to life. The award is USD100K. Note that we are NOT interested in on creating the world's best cheaters, ie. a chatterbot working with hand tuned rules, phrases, etc. That's not COOL, IMHO.

Biometrics

• Signature Verification Competition. Given a bunch of signatures (either on-line/dynamic or off-line/vision based) signatures of a person, and given a new signature determine of the given new signature belongs to the same person. For the dynamic version a special pen is used that records the signature as a multi-dimensional time series data. For the static/off-line/vision-based version the input is a bitmap. So these two tasks actually require quite different approaches. Or you think not?

• Fingerprint Verification Competition. Same as above, now you have bitmaps of fingerprints. 

• Face Verification Contest.

• Speaker Recognition/Verification - see the speech section below.

Vision

• ICDAR 2003 competitions. These are vision based text processing tasks such as e.g. cursive script recognition, page segmentation, table segmentation, robust reading, robust word recognition, robust character recognition, text locating. Out of these we are interested in the robust stuff and locating text (we also have some experience on tasks like these!).

• See above, the signature fingerprint, face verification competitions.

• FERET face recognition competion

Speech

• NIST Speaker Recognition Benchmark

• See also NIST's benchmark page below.

Other

• IAPR TC-5 Benchmarking and Software - this is a starting point for many competitions (most listed above). Also lists upcoming competitions and has pointers to valuable datasets (e.g. MNIST, USPS). Competitions of interest include e.g. 

• NIST organized "evaluations", some of these are listed above.

• Brain Computer Interfaces Just added here, because found this page. No experience on tasks like this!

• PASCAL network ("pattern analysis, statistical modelling and computational learning as core enabling technologies for multimodal interfaces that are capable of natural and seamless interaction with and among individual human users") - this is an EU network that organizes challenges regularly. Here is the recent list. Textual entailment ("Textual entailment recognition is the task of deciding, given two text fragments, whether the meaning of one text is entailed (can be inferred) from another text") sounds pretty interesting.