Data Model for Data Science Model Performance Evaluation

The Data Scientists need to experiment with models, with feature engineering , with type of model – traditional or deep learning, before making a recommendation with a story to the decision makers.

It is also important in case of models such as Recommendation Systems or Value Estimations to be able record their performance and also the actual events thereof so as to know what needs to be done in future.

A data model which stores relevant data about the data science activity from start to finish of the model lifecycle would help.

There are two reasons why storing this data and analyzing to make improvements make sense.

One, as opposed to a deterministic programming model, a stochastic model may have many options in terms of algorithms. Since in a machine model, the test is only acceptability rather than correct or incorrect, the improvements are almost always possible.

Second, as business conditions and other things impacting the data used as input change, the impact to the output may be more sudden and adjustments either necessary or simply urgent.

I like the paradigm. You use machine learning models to provide diagnosis and suggestions for actions. Then you create machine learning models on these models to do a second order analysis and hence, take action on your original pipeline.


Table – Model Performance

Date Timestamp

Data Set Id

Pipeline Id

Model Id

Hyperparameter

Metrics


Table – Pipeline

Pipeline Id

Pipeline Step

Pipeline Step Description


Table – Model Type

Model Type

Model Type Description


Table – Model

Model Id

Model Name

Model Description

Model Type


Table – Raw Dataset

Raw Dataset Id

Raw Dataset


Table – Final Dataset

Final Dataset Id

Final Dataset


Table -Final Dataset Feature

Final Dataset Id

Feature Id

Feature Type

Feature Origin Type

Feature Name

Feature Description

Valid Values


Table- Feature Instance Prediction

Feature Value List

Model Id

Prediction

Actual Value

Impact


With thanks to

Lynn Langit
Srinivasan Srivatsan

Sponsored by:

WordPress

One thought on “Data Model for Data Science Model Performance Evaluation

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.