Good Representation Can Ease Learning


It appears we seem to spend lot more time on picking a learning methodology and ignore pattern representation and feature representation. This generally comes at a cost in terms of more training time and predictive accuracy. I thought I will highlight this practice by showing a very simple example. Slide #1 shows a set of training examples and the corresponding label associated with each example. The task is to learn the labeling pattern so as to able to assign labels to test examples shown in the slide.

image

Some of you will try to unlock the labeling pattern by looking at the data because we have few examples only. Some of you with access to machine learning software will immediately plug in these training examples and select their favored method of learning, for example decision trees, to figure out the relationship between the numbers and their labels.

Let us look at our problem again. But this time let us use a different representation for our problem as shown in Slide #2.

image

In this case, you will be easily able to discern the labeling pattern as shown in Slide #3. The label assigned to a number depends upon the logical AND of second and sixth bits of binary representation of each number. Thus, it is useful to remember that a good representation, the choice of features to represent patterns, makes learning easier.

image

Advertisements

Author: Krishan

I am a professional with many years of experience in computer vision, data mining, machine learning, neural networks, and pattern recognition. I provide consulting and training services through my company, Integrated Knowledge Solutions.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s