While Concepts in WEM is a very useful and powerful feature, it can also cause some difficulty in understanding certain situations in Runtime. We hope to clarify some here.
First of all, know that Modeler and Runtime are two physically and logically separated environments (you can read about them in other areasof this documentation). The Modeler holds all your project definitions and the logic, the Runtime receives the published result of that definition when you run the publish process and transforms the Model-definition into a working application for your end-users.
The definition of Concepts (their properties and hierarchy) exists within the Modeler, in the definition of your project. When published, the necessary information is sent to the Runtime Application.
That Runtime environment has two parts (simplified; actually there are more, but for this situation only these 2 parts need to be considered):
the Runtime Application serving your project to end-users;
the Runtime Database storing the values;
When you publish your project to the Runtime, the project-definition gets send to the Runtime Application so your project can be used as application. The definition of the persistent lists and fields in your Project will be forwarded to the Database server, to update the database structure to store the persistent values.
What needs to be understood, is that the definition of the concepts is NOT part of the Runtime Database definition but is part of the Runtime Application definition, so to be clear: the Database environment does NOT know about the concept definitions. Persistent fields based on concepts will only store the numeric ID-values of those concepts which are defined in the Modeler environment. The database can only recognize this value as a numeric field - nothing more, nothing related to the concept definition. So, when this numeric value is retrieved from the database in any way, it is nothing more than that number, not until it gets combined with the Project Definition on the Runtime Application environment.
What happens when you remove an existing and previously used concept from the definition, and publish that change to the Runtime:
Only the Runtime Application is updated, because this is a change in definition not affecting the database structure;
Any Database fields that have stored the deleted concept's numeric ID, are NOT cleared automatically - because it is only a numeric value without any meaning to the database context itself (no foreign keys, no concept-definition);
When this field's value is used in the Application, the old stored ID can no longer be matched against the valid collection of concepts and will be cleared if that record is saved in the process. So while the field may appear empty in the Runtime Application when looking at the record, the field in the database still holds the old numeric value until it is updated with a new value (or cleared) following a save action from the Application;
When this field is retrieved or checked using Database Select or Filter expressions, the execution of the expression will see this field as containing a numeric value, so a check with IsEmpty() or IsUnknown() will return false. Also, data retrieved by external application directly via OData, will also return the old numeric value.
As Modeler of the project, you should be aware that when you remove concepts that have been in use in the Runtime so they may have been stored in Runtime Database, you should make the appropriate actions to handle the removal of the concept.
One option would be, to create a flow that checks all rows in a list where the deleted concept may be used, and update the field. Many WEM Consultants are accustomed to create certain "admin fixes" flows that need to be run (by admin) in runtime after a certain change has been made and published.
Another option would be, to use the following expression when you are thinking of using a filter expression checking for IsEmpty([concept]) when you want to find deleted concepts (which as you now understand, are not empty):
Range of is an expression that gets the list of available concepts based on the field definition of the concept-field (including depth and type settings), which - in this case of a deleted concept - will no longer contain the deleted concept, and the stored numeric value in the database will therefore no longer match. This statement is NOT SQL Compatible because of the same reason: the definition of the concepts is NOT part of the Runtime Database, so using this statement will require to load all records in memory and check each record on the application server in current session where the data can be combined with the portal definition containing the concept definition. This will make the expression slower in performance as compared to the SQL Compatible IsEmpty() function, but we hope the reason for this is now clear enough.