Spaces:
Sleeping
Sleeping
Commit
ยท
9aa1348
1
Parent(s):
bee26ea
Fix links
Browse files
app.py
CHANGED
@@ -6,13 +6,13 @@ st.title('Is your model degrading?')
|
|
6 |
st.caption('### :violet[_Estimate_] the performance of an ML model. :violet[_Without ground truth_].')
|
7 |
|
8 |
st.markdown("""
|
9 |
-
If you have been previously exposed to concepts like [covariate shift or concept drift](
|
10 |
you may be aware that changes in the distribution of
|
11 |
the production data can affect the model's performance.
|
12 |
""")
|
13 |
|
14 |
st.markdown("""Recently a paper from MIT, Harvard, and other institutions showed how [91% of their ML models
|
15 |
-
experiments degraded](
|
16 |
|
17 |
st.markdown("""Typically, we need access to ground truth to know if a model is degrading.
|
18 |
But most of the time, getting new labeled data is expensive, time-consuming, or impossible.
|
@@ -45,7 +45,7 @@ will continue to be good on production data?
|
|
45 |
st.markdown("#### Estimating the Model Performance")
|
46 |
st.markdown("""
|
47 |
Instead of waiting for ground truth, we can use NannyML's
|
48 |
-
[CBPE](
|
49 |
method to estimate the performance of an ML model.
|
50 |
|
51 |
CBPE's trick is to use the confidence scores of the ML model. It calibrates the scores to turn them into actual probabilities.
|
@@ -103,6 +103,6 @@ st.divider()
|
|
103 |
st.markdown("""Created by [santiviquez](https://twitter.com/santiviquez) from NannyML.""")
|
104 |
|
105 |
st.markdown("""
|
106 |
-
NannyML is an open-source library for post-deployment data science. Leave us a ๐ on [GitHub](
|
107 |
-
or [check our docs](
|
108 |
""")
|
|
|
6 |
st.caption('### :violet[_Estimate_] the performance of an ML model. :violet[_Without ground truth_].')
|
7 |
|
8 |
st.markdown("""
|
9 |
+
If you have been previously exposed to concepts like [covariate shift or concept drift](https://www.nannyml.com/blog/types-of-data-shift),
|
10 |
you may be aware that changes in the distribution of
|
11 |
the production data can affect the model's performance.
|
12 |
""")
|
13 |
|
14 |
st.markdown("""Recently a paper from MIT, Harvard, and other institutions showed how [91% of their ML models
|
15 |
+
experiments degraded](https://www.nannyml.com/blog/91-of-ml-perfomance-degrade-in-time) in time.""")
|
16 |
|
17 |
st.markdown("""Typically, we need access to ground truth to know if a model is degrading.
|
18 |
But most of the time, getting new labeled data is expensive, time-consuming, or impossible.
|
|
|
45 |
st.markdown("#### Estimating the Model Performance")
|
46 |
st.markdown("""
|
47 |
Instead of waiting for ground truth, we can use NannyML's
|
48 |
+
[CBPE](https://nannyml.readthedocs.io/en/stable/tutorials/performance_estimation/binary_performance_estimation/standard_metric_estimation.html)
|
49 |
method to estimate the performance of an ML model.
|
50 |
|
51 |
CBPE's trick is to use the confidence scores of the ML model. It calibrates the scores to turn them into actual probabilities.
|
|
|
103 |
st.markdown("""Created by [santiviquez](https://twitter.com/santiviquez) from NannyML.""")
|
104 |
|
105 |
st.markdown("""
|
106 |
+
NannyML is an open-source library for post-deployment data science. Leave us a ๐ on [GitHub](https://github.com/NannyML/nannyml)
|
107 |
+
or [check our docs](https://nannyml.readthedocs.io/en/stable/landing_page.html) to learn more.
|
108 |
""")
|