Tuesday 2 June 2009

Information IS data in context

There has been a lot of discussion about the meaning of data, information, knowledge, wisdom and metadata. Though often entertaining this discussion usually generates more light than heat. Here I will use an expanded version of the semiotic framework to resolve the confusion. I find that information IS "data in context" and data IS an encoded form of information. However, the context has several layers, each of which contributes to the encoding of information as data. Much of the confusion is due to ignoring the layered nature of the context.

I will consider the steps needed to fully understand a message (or stored record). I’ll assume that the message is digitally encoded – though this is almost irrelevant to the analysis. Initially, let’s assume that the message is English text.

a) I receive a message comprising a string of bytes.

b) I can covert this into printed, or displayed, characters. But to do this properly I must know which character coding system, eg EBCDIC, was used to create the message.

c) Next I use my lexical knowledge to recognise words and punctuation marks.

d) And then my syntactic knowledge to parse it. I now see the text as a structure, specifically a sentence, with a subject, clauses, etc.

e) Adding my semantic knowledge (of what the words mean) gives me the meaning of the sentence.

f) Finally I relate this meaning to other relevant knowledge (the context for the sentence) and see the significance of the message.

At each step the decoder, whether human or electronic, must apply information it already knows. This information may be called metadata. Here’s a summary:

Semiotic level

What is added by the receiver

The result

6 Pragmatic

Contextual knowledge.

Significance of the information.

5 Semantic

Meanings of words (from dictionary)

Meaning of the sentence

4 Syntax

English grammar

Parsed sentence.

3 Lexical

Lexical rules

Words

2 Coding

Character coding.

Characters

Input

n/a

Bytes

Now let’s generalise.

Numbers

Suppose the message consists of several distinct fields, some textual and some numerical. Then we’ll need to divide the message into fields before we apply the character code for text since numbers may use non-text coding. This will be level 1. And we’ll still need level 2 to turn the bytes into numbers.

There’s no lexical or syntactic level for numbers but we will need to know the semantics. Many numbers are measurements or predictions of measurements and for these we need to know what is measured and the units. We may even need to know how the measurement was made and by whom.

Other numbers are ratings or rankings and this also needs to be known.

Some of this may, of course, be given explicitly by other data items. Taken together this information allows us to convert the number into a sentence of known meaning, eg

  • 17.63 => The height of the mast is 17.63 m.
  • 623 => This MP’s expenses were the 623rd largest.

Finally, at the pragmatic level, we add contextual information to see the significance of the information, eg

  • The boat is too tall to pass under the bridge.
  • This MP is probably honest.

Image

Now suppose that we have fields containing image data. Processing is similar to numbers. The image coding, eg JPEG, is used at level 0 and there is no lexical or syntactic processing. To get the meaning of the image (semantic level) we need to know what sort of image it is, eg an aerial photograph or an X-ray image, its scale and perhaps other details about the equipment and settings used.

Finally, as before, the pragmatic level adds contextual information to let us see the significance of the image.

Composite model

We can put all this together in a composite model as shown in the table. If necessary the Images column can be generalised to cover the result of any sensing system, eg video, radar images, seismograph output.


Text

Numbers

Images

6 Pragmatic

Add contextual knowledge to get significance.

5 Semantic

Add dictionary knowledge to get meaning.

xx

And units.

And scale, part of spectrum sensed.

4 Syntax

Add NL grammar to get a parsed sentence.

xx

xx

3 Lexical

Add lexical rules to get words

xx

xx

2 Coding

Add character coding to get characters.

Add format rules to get numbers.

Add format rules to get 2D image.

1 Record structure

Add record structure knowledge to divide message into fields.


Bytes

I started by noting the muddle about the words data, information, etc. In fact these words are often used interchangeably and in different ways by people with different backgrounds and interests. IT people, however, should say ‘data’ when we want to discuss bits and bytes and their decoding and processing at lexical and syntactic levels. We should say ‘information’ when discussing the semantics and significance of the data.