SQL Server 2005 Limitation: Having many attributes for Neural Network

MaryClinton


Hello,

Say that I have 100,000 attributes/feature selections for my SQL Server Neural Network Algorithm.

Customer Attr1 Attr2 Attr3 ..... Atr100000
==============================
Jack 1 0 1 ..... 1
Sam 0 1 1 ...... 0
Mary 1 1 0 ...... 1

Knowing the fact I can't fit those info on a table and SQL Server's Neural Network does not support table prediction . What's an alternative to use Neural Network in SQL Server 2005 to solve my problem

Please assist!


Mary

Jamie MacLennan

The reason that Neural Networks doesn't support table prediction is precisely to avoid situations where you could create models to predict 100,000 attributes. Essentially what you are asking the model to do is to create 100,000 simultaneous neural networks. Each of these networks would have 100,000 input nodes, somewhere on the order of 100,000 squared hidden nodes and one output node. A few calculations would determine that the memory required to solve such a problem would be enormous, and you likely wouldn't want to wait until it completed even if you had the memory.

For example, take your 100,000 attribute problem as input and predict a case-level attribute. Time that and multiply by 100,000 (if you have a dual proc box, divide by 2!). Yes, that's how long it would theoretically take assuming you had enough memory.

Let me know if you disagree that the problem you are trying to solve isn't likely suitable for NN. I would try Naive Bayes or Decision Trees for this problem. Logistic Regression may also work, but I'm unsure of it's nested table requirements.

Bogdan Crivat

Logistic Regression cannot handle nested table predictions either. Do all the attributes have to be predictable

MaryClinton

No, I only care the attributes with value 1 on each row.

For example:
Jack = Attr1, Attr3, Attr100000
Sam = Attr2, Attr3
Mary = Attr1, Attr2, Attr100000

Customer Attr1 Attr2 Attr3 ..... Atr100000
==============================
Jack 1 0 1 ..... 1
Sam 0 1 1 ...... 0
Mary 1 1 0 ...... 1


To give a better idea of my problem. Here is the scenario.
Say that I have a set words to describe each customer's characteristics.

Customer Friendly Tall Rich ..... Good Looking
===================================
Jack 1 0 1 ..... 1
Sam 0 1 1 ...... 0
Mary 1 1 0 ...... 1


Jack = Friendly, Rich, Good Looking
Sam = Tall, Rich
Mary = Friendly, Tall, Good Looking

I also have 100,000 attributes (Friendly, Tall, Rich, ...., Good Looking). Each attribute can only have 1 or 0 value.

Say that I have 1 million rows.
In some cases, the same customer may appear more than once in the table.
For example:
Simon 0 1 1 ...... 0
Simon 1 0 1 ...... 0


I only care the attributes with the value of 1.
Later, I want to use Neural Network to find the customers who are friendly and rich.

What the best way to implement such idea in Neural Network Please assist!


Regards,
Mary

Jamie MacLennan

You should make the attributes you want to predict case level attributes and put all of the other attributes in a nested table which will be input only. For example "Friendly" and "Rich" will be case level attributes with values 0 or 1 and the other attributes will just be nested keys, e.g. something like this.

CREATE MINING STRUCTURE DatingStructure

(

[UserID] LONG KEY,

Friendly LONG DISCRETE,
Rich LONG DISCRETE,

Attributes TABLE

(

Attribute TEXT KEY

)

)

ALTER MINING STRUCTURE DatingStructure
ADD MINING MODEL DatingModel

(

UserID,

Friendly PREDICT,

Rich PREDICT,

Attributes (Attribute)

) USING Microsoft_Neural_Nets

You will need two tables - one that has the UserId and the first two columns for each case, and the second that is a transactional table containing a column for the UserId and the Attribute name. Note that if you have the same user twice, you will need a different userid for each one to represent that user. Eg. Simon1 and Simon2 or whatever. Usually you can make a compound key by appending two columns (e.g. the name and the date). The only requirement is that the UserID in the case table and the UserID in the nested table match.