An attribute is, abstractly speaking, a piece of information about an entity. For example, an EOEntity may represent a person. The first name of the person is an attribute. The zip code of the person's address may be an attribute. An attribute is represented in EOF as a com.webobjects.eoaccess.EOAttribute instance. If an entity's information comes strictly from a table, then an attribute will be the object representation of a column. A relationship in an entity may also be an attribute of the entity.
A "class" EOAttribute" is distinguished from a non-class EOAttribute by whether or not the attribute is supposed to be an attribute of the class which models it. For example, if you have the first name of a person, that would be a class attribute. If you have an attribute you use in the entity to act as a key for a join, that attribute itself is not part of the class as such. It is part of the machinery used by the class. That key, itself, would not be something that one would want a user to know about. If that attribute is used in a join, and that join defines a relationship, then one would want to expose the relationship to a user. The relationship would be a class EOAttribute and the key in the join would be a non-class EOAttribute.