Skip links
Main content

The problem with proper nouns

zaterdag 19 mei 2012 09:32

Before a sentence is passed to the parser, it is first run through the lexer. The lexer divides the sentence into words or word groups. These are called lexical items and they are the smallest parts of the sentence that have a single part-of-speech. Most lexical items can be looked up in the lexicon. But what about proper nouns?

Proper nouns (or more exactly: proper names), lexical items like "Sandra", "Peter R. de Vries" and "Ramachandra" are not part of the lexicon (dictionary). Therefore they will not be recognized simply by doing a lexicon lookup.

Still, you may be able to recognize some of them by looking them up in a knowledge base.

When was Lord Byron born?

The name "Lord Byron" may be looked up and form a single lexical item with the part-of-speech "proper noun".

[when, was, Lord Byron, born]

But this sentence is different:

My name is Jan de Vries.

The name of the conversation partner cannot be looked up. And this is only logical, because the purpose of the sentence is introduction. And I mean that in two ways: the person introduces himself. And at the same time a new lexical item, "Jan de Vries", is introduced in the mental lexicon of the agent.

How can a lexer recognize a new proper noun when it encounters one? I can think of two ways:

1. The words in the sentence that are not part of the lexicon form the new proper noun.
2. Proper nouns may be recognized because they follow certain patterns.

The first method is simple and applies to all names. However, it runs into several problems.

For example:

My name is Jan de Vries.
[my, name, is, Jan, de, Vries]

Since "de" is a word in the lexicon, are "Jan" and "Vries" to be separated into two separate proper nouns? No, because we want to end up with single lexical item. So we may want to take all words in between the unknown words as well. However, which words should we include and which words should we not?

My name is Jan and I like Sandra.
[my, name, is, Jan and I like Sandra]

Why can "Jan and I like Sandra" not be a proper noun? Because it is too long? What about

My Name is Johannes Hubertus van der Laak.

It contains just as many words.

That brings us to the second option: patterns.

The pattern for "Jan de Vries" is 

"Ul+ de Ul+" (Ul+ = upper case letter followed by one or more lower case letters)

and the pattern for "Johannes Hubertus van der Laak" is

"Ul+ Ul+ van der Ul+"

Why require the capitals? People often leave them out, don't they? Yes they do. However, if the restriction on capital letters is dropped, the following sentence

john mills the grain.

would be lexed as

[john mills, the, grain]

and this cannot be parsed.

Remember, the lexer has no knowledge of the syntactic structure of the sentence. If the lexer was integrated in the parser, it might be more flexible and offer different possible interpretations. However, this makes the parsing process more complicated and computationally expensive.

References

Labels
nlp

« Terug

Reacties op 'The problem with proper nouns'

toms shoes
Geplaatst op: 26-06-2013 15:42 Quote
Howdy! This article could not be written any better! Reading through this post reminds me of my previous roommate! He continually kept preaching about this. I'll send this post to him. Fairly certain he's going to have a very good read. Many thanks for sharing!| toms shoes http://www.ideavelopers.com/tomsshoes.php
louboutin
Geplaatst op: 02-07-2013 06:32 Quote
These are genuinely wonderful ideas in regarding blogging. You have touched some good things here. Any way keep up wrinting.<a href="http://www.ideavelopers.com/tomsshoes.php">www.ideavelopers.com/tomsshoes.php</a>| louboutin http://www.ideavelopers.com/louboutinfr.php
discount nfl jerseys
Geplaatst op: 17-09-2013 19:18 Quote
999 Recall Maks Erin and Maks Brandy and Maks Everybody under the sun Else?. The new York Moments has thrown us an alternate curve this 7 days.
discount nfl jerseys
babyliss pro perfect curl
Geplaatst op: 17-09-2013 19:52 Quote
999 Recall Maks Erin and Maks Brandy and Maks Nearly everybody Else?. The new York Days has thrown us a further curve this week.
babyliss pro perfect curl
cheap sports jerseys from china
Geplaatst op: 28-09-2013 19:39 Quote
999 On the past couple of several years, cellular phone equipment are getting to be more and more attractive. This really is not about throw it for the wall and check out it stick.
cheap sports jerseys from china
christian louboutin discount shoes
Geplaatst op: 16-10-2013 16:11 Quote
333 Put the price of a $100 enjoy as much as $150 and prospects will change absent. When you are it authentic the relatively easy gladness of a
christian louboutin discount shoes
louis vuitton purses on sale
Geplaatst op: 16-10-2013 18:22 Quote
333 Set the cost of a $100 see as many as $150 and patrons will change absent. While you are it a fact the uncomplicated gladness of a
louis vuitton purses on sale
cheap soccer jerseys
Geplaatst op: 17-10-2013 15:27 Quote
333 Put the cost of a $100 view approximately $150 and customers will transform away. While you're it legitimate the easy to understand gladness of a
cheap soccer jerseys
jerseys cyber Monday
Geplaatst op: 17-10-2013 16:13 Quote
333 Put the cost of a $100 enjoy as much as $150 and prospects will change away. While you're it genuine which the effortless gladness of the
jerseys cyber Monday
christian louboutin outlet
Geplaatst op: 17-10-2013 16:53 Quote
333 Put the cost of a $100 check out about $150 and potential clients will flip away. If you are it genuine which the very easy gladness of the
christian louboutin outlet
classic ugg boots on sale
Geplaatst op: 18-10-2013 16:09 Quote
999 The invention of the new antitheft not merely breathtaking but additionally easy. She would not chat carats or price tag tags even though demonstrating me a hoop very similar for the a particular Ponder procured that was specially designed by Precision Established.
classic ugg boots on sale
christian louboutin discount mens
Geplaatst op: 19-10-2013 16:16 Quote
888 If learn how to put on saree is complex question for yourself then below you would probably find the rather simple solution to this issue.. She was magnificent in her 20s, she's beautiful now and she'll be magnificent twenty years from now.
christian louboutin discount mens
mens louis vuitton wallets
Geplaatst op: 19-10-2013 16:19 Quote
888 If proven methods to don saree is sophisticated concern for yourself then right here you would find the basic reply to this issue.. She was lovely in her 20s, she's magnificent now and she'll be beautiful 20 years from now.
mens louis vuitton wallets
buy discount christian louboutin shoes
Geplaatst op: 21-10-2013 13:17 Quote
888 If easy methods to have on saree is tricky problem to suit your needs then listed here you'd locate the hassle-free answer to this query.. She was magnificent in her 20s, she's gorgeous now and she'll be lovely 20 years from now.
buy discount christian louboutin shoes
louis vuitton handbags outlet usa
Geplaatst op: 23-10-2013 08:58 Quote
9999 The tissue around the creases can help to cut back the tension but it surely cannot stop it. My cruisemate hadn't been, but feels he doesn't should return..
louis vuitton handbags outlet usa
cheap bengals jerseys
Geplaatst op: 24-10-2013 14:01 Quote
999 "I hope my daughter won't browse through this!" claims the mom of Lily.. The Damier Canvas Speedy 30 can be described as relatively popuar bag in Big apple .
cheap bengals jerseys
authentic vintage louis vuitton handbags
Geplaatst op: 28-10-2013 17:36 Quote
3333 Marc has famously battled drug habit inside of the previous and has common remedy classes. That is certainly why in spite of the shaky economic local climate, Zara's strategy for society domination just isn't faltering.
authentic vintage louis vuitton handbags
quality replica louis vuitton belts
Geplaatst op: 22-11-2013 05:27 Quote
999 The invention of the new antitheft not simply breathtaking and also handy. She would not communicate carats or expense tags whilst exhibiting me a hoop equivalent with the one particular Ponder purchased which was engineered by Precision Set.
quality replica louis vuitton belts
louis vuitton replica luggage uk
Geplaatst op: 22-11-2013 05:32 Quote
999 It's really a cologne produced for men somewhere between the ages of 35 and 53, and will be worn while in the daytime. I'd carry on to spend given that likely chopping it up accompanied by a fellow geek, the many even as not providing everything.
louis vuitton replica luggage uk
real louis vuitton purses for sale
Geplaatst op: 22-11-2013 05:54 Quote
999 It is a cologne established for guys concerning the ages of 35 and 53, and should be worn all through the daytime. I would commence to spend given that quite possible chopping it up that has a fellow geek, all of the as not offering everything.
real louis vuitton purses for sale
cheap dakota uggs
Geplaatst op: 22-11-2013 12:21 Quote
999 The creation of this new antitheft don't just stunning but in addition easy. She would not discuss carats or total price tags while you are displaying me a ring similar for the an individual Ponder ordered that was developed by Precision Established.
cheap dakota uggs
Nieuw bericht