Welcome back to the Deep Dive. Today we are wading into waters
that feel, honestly, a little dangerous.
Dangerous. That's a strong way to start.
Well, think about it. We're talking about high stakes
medicine. We're talking about diagnosing
rare legal diseases, situations where accuracy is literally life
or death. And the premise of the research
we're looking at today is that the best way to save those lives
is to use, well, fake data. But that way it does sound a bit
like malpractice, doesn't it? Right.
I mean, if I'm a patient and I hear my doctor is training their
diagnostic AI on patients that you know don't even exist, I'm
getting a second opinion immediately.
And you'd. Be right to be skeptical, but
this is the big butt for today. What if I told you that relying
only on real patient data is actually what's making the AI
fail? How so?
That the real data is, in a way, mathematically broken when it
comes to the sickest people. That is the paradox we are
unpacking today. This is our Practical AI Digest
edition. So for all of you out there who
are building these systems, this one's for you.
We are looking at a stack of research, primarily a new
technical paper from Scientific Reports that was validated on
COVID-19, kidney disease and Dengo datasets.
A pretty serious lineup of diseases.
Exactly. And we're also pulling in some
industry specs from Blue Gen. AI and an explainer from IBM
Technology. But we aren't just talking
theory here. We are breaking down a specific
buildable framework. That's the key.
We're looking at a hybrid pipeline.
We're going to talk about combining classical statistical
tricks with some some really heavy hitting deep learning.
Specifically. Specifically Deep CT, Jan and
something called residual networks or resnets.
And just to really set the stage for you builders listening, by
the end of this, we want you to walk away with the recipe.
We're going to cover the architecture, the specific hyper
parameters. I'm talking learning rates,
batch sizes, the works, and how to validate it so you don't end
up, you know, just hallucinating patient data.
It's basically a master class in handling the imbalance data set
problem. So let's start there.
The intuition You mentioned that real data is broken or biased.
I think most people assume data is neutral.
It's just numbers in a spreadsheet.
How can a spreadsheet have a bias against sick people?
It's not a bias in the human sense.
It comes down to the math of optimization.
We call it majority class bias. OK, think about a typical
hospital database. Let's say you pull records for,
I don't know, 1000 patients, OK. 1000 rows in my CSE file.
I've got it open. Now, how many of those thousand
people actually have a rare, specific form of kidney disease?
Very. Few I'd imagine.
Maybe 50, maybe even fewer. Right.
The vast majority are there for check UPS or broken arms or the
flu. They don't have this specific
critical condition. Exactly.
So you have a 95 to 5 split, 950 healthy people, 50 sick people.
Now you feed this to a standard machine learning classifier, say
a random forest or a basic neural net.
The models only goal is to maximize its accuracy.
It just wants to get the highest score possible.
That's it. And that's where the trap is.
The model quickly realizes, wait a minute, if I just predict
healthy for every single person, I'm right 95% of the time.
It's the lazy student strategy. It crams for the easiest part of
the test and aces it, but it doesn't learn anything.
Precisely. In a computer science class, 95%
accuracy is amazing. You get an A.
But in an oncology ward, that model is a catastrophe.
It's worse than useless. It has 0% sensitivity.
It missed every single cancer case.
It maximized its score by completely ignoring the minority
class. So the data itself isn't
prejudiced in a way we'd recognize, but the distribution
of the data forces the algorithm to ignore the edge cases.
And in medicine, the edge cases are usually the ones that need
saving. This is why we need synthetic
data. We have to artificially inflate
that minority class so the model can't just brush it aside
anymore. The IBM source had a really good
analogy for this. I thought they used football.
The Premier League one, right? Yeah, that's a great way to
think about. It walk us through it.
OK, so if you trained an AI on the last 20 or 30 years of
English football and and asked it to predict the next winner,
what would it say? Manchester City, Manchester
United, Chelsea. One of the big rich clubs,
right? It creates a simple rule in its
head. Rich big teams win.
It's seen that pattern over and over.
It's the majority class. And then 2016 happens.
And Leicester City wins the whole thing.
The ultimate outlier, a total statistical anomaly.
Exactly. If your model is only looking at
the majority history, it says that event is impossible, but it
happened. Synthetic data lets us simulate
thousands of Leicester City seasons.
It lets us create these what if scenarios that force the model
to learn the characteristics of the underdog.
Or, in our case, the characteristics of the rare
disease. That's the perfect parallel.
We aren't just copying data, we are creating new examples that
follow the same underlying rules of physics and biology, but they
fill in the gaps where a real data is thin.
So it's crucial to understand we aren't just typing in random
numbers. Oh, absolutely not.
That would be useless. We're learning the statistical
distribution, the hidden shape of the data, and then we're
sampling from that shape to create new plausible points.
Now before we get to the how, I want to clarify the use case
because the blue Gen. AI material mentions 2 main
reasons for using synthetic data.
One is privacy, which is massive.
Huge. Yeah.
GDPRAPA, right? If the patient is completely
fake, you can't get sued for leaking their data.
You can share it with researchers freely.
True, that's what we call the replacement use case.
But for this deep dive, for what this paper is about, we are
focused on augmentation. OK, what's the difference?
We aren't throwing away the real data.
We are supplementing it. We are taking those 50 real sick
patients and generating say 450 new synthetic sick patients.
So you turn the 50 into 500. And suddenly the lazy student
can't get an A by ignoring them. They have to actually study.
OK, let's get into the engineering.
This hasn't been a straight line to success.
There's been an evolution here, a couple of tiers of technology.
A pretty clear one actually. It's really been a journey from
simple geometry to complex deep learning.
We can call tier one classical resampling.
This is stuff you could run on a laptop back in 2010.
Pretty much the big name here, the one everyone learns, is
SOBT. SMOTE Synthetic minority
oversampling technique. It's a staple in data science
interviews. It is, and it's intuitively
very, very simple. Imagine your data points are
plotted on a 2D graph. You have your big cluster of
healthy patients and then you have this little cluster of 50
sick patients floating off in a corner.
OK, I can picture it. SMODI picks one of those sick
patients at random, then it finds its K nearest neighbors,
so the other sick patients that are closest to it on the grass.
And then it just connects the dots.
Literally it draws a straight line between the patient it
picked and one of its neighbors, and then it just picks a random
spot on that line and says new patient here.
So it's just linear interpolation.
It's all it is. So if patient A has high blood
pressure and patient B who's nearby has slightly higher blood
pressure, smell may just assumes A valid patient C exists
somewhere in the middle. Precisely.
The math is just X nu plus Lambda X neighbor a she.
It's simple, and for some problems it works surprisingly
well, but there's a massive flaw.
Biology isn't a straight. Line lingo biology is messy.
It's nonlinear. Complex diseases don't always
follow a straight, predictable path between two patients.
If you just draw lines, you might be creating patients that
are biologically impossible. Or just not very representative
of the true complexity, right? Plus, COT is kind of dumb in how
it samples. It treats every minority point
the same. It doesn't care if a data point
is easy to classify or really confusing and sitting right on
the fence. And that's where the upgrade
came in, right? Ada sign.
ADA sign hole adaptive synthetic sampling.
This is like SMOTE scene with a bit of a brain.
It looks at the data set and asks where's the model
struggling the most. It hunts for the hard to learn
examples. Yes.
It looks at each minority point, each sick patient, and counts
how many of its neighbors are from the majority class, the
healthy patients. So if a sick patient is totally
surrounded by healthy patients, it's.
In what we call a hostile environment, it's an ambiguous
case. Adis YN sees that and assigns a
higher weight to that point. It says we need more troops here
on the frontline. So it generates more synthetic
data specifically along that messy decision boundary to try
and clear up the confusion. Exactly.
It's much smarter, but it's still geometric.
It's still just drawing lines. And that brings us to the limit
of Tier 1. It does when you have high
dimensional data. We're talking hundreds of
columns with gene expressions, complex symptom interactions,
lab results. Simple geometry just breaks down
completely. You can't draw a line in 500
dimensional space and expect it to mean anything.
You need something that can learn the manifold.
The curved, twisted shape of the data's probability distribution.
You need deep. Learning and enter the Jans
generative adversarial networks. Tier 2, the famous cat and mouse
game. Or maybe artist and critic is a
better way to put it. Break it down.
You have two neural networks fighting each other.
The generator is the artist. It tries to create a fake
sample, like a fake patient record.
The discriminator is the critic. It looks at the fake record
alongside a real 1 and has to guess which is which.
And if the discriminator spots the fake.
The generator gets punished, gets a bad signal and has to go
back and try harder next time it learns from its mistake.
And this goes on and on for. Thousands of rounds until
eventually the generator is so good at creating fakes that the
discriminator is no better than guessing its accuracy is 50%.
And at that point you have a master forger, you have high
fidelity synthetic data. That's the theory, but here's
the big snag. Standard jams are terrible at
sreadsheets. Why?
I mean, they can generate photorealistic faces of people
who don't exist. Why is a spreadsheet of patient
data harder than a human face? It's a great question.
It's because of the data types. Pixels in an image are all more
or less the same kind of thing. They're continuous values from
zero to 255. They flow into each other
smoothly. But a spreadsheet, it's a total
mix. You have a column for blood
pressure, which is a continuous number.
From right next to it is gender, which is a discrete category.
Then blood type another category.
This mix of discrete and continuous data completely
confuses standard Jans. And the distribution of those
numbers is usually really weird, right?
Like blood pressure isn't just one simple bell curve, it might
have a couple of different peaks for different patient groups.
Exactly. It's multimodal, yeah, and
standard Jans suffer from something called mode collapse.
Mode collapse. They find one type of easy to
fake patient one mode in the data and they just keep
generating that same kind of patient over and over again.
They ignore all the variety and especially the rare cases.
Which defeats the entire purpose of what we're trying to do.
We need the rare stuff. That is precisely why the
Scientific Reports paper didn't just use a vanilla Jan, they
used something called Deep CT Jan.
OK, let's breakdown that acronym.
CT Jan. Right, so the T is for tabular
because it's designed for spreadsheets.
The C is the real magic here. It stands for conditional.
Conditional. And that lets us explicitly tell
the generator what we want. We can give it a condition, we
can say I want you to generate a sample, but only for the
minority class. So instead of just hoping it
stumbles upon making a sick patient, you can actually force
its hand. You condition the input, you say
give me a plausible breakdown of a patient with dengue fever, and
this forces the model to learn the specific complex
distribution of that class, which is what prevents mode
collapse. So that's the engine, the deep
C2 Gan, but the paper we're discussing today proposes a
hybrid framework. They didn't just turn on the Jan
walkway, they built a whole pipeline around it.
This is the real how to section for the engineers listening.
If you're building this, you need to follow these three steps
they laid out. OK, Step 1, preprocessing.
This is standard data hygiene. You can't train a model on empty
cells for any missing numbers that used mean imputation.
For missing categories they used mode imputation.
So fill in the blanks with the average or the most common
value. Nothing fancy, but essential.
Absolutely essential garbage in garbage.
Out Step 2 is interesting. They call it initial balancing.
This is a really clever move. Before they even touched the big
expensive deep learning model, they used the classical methods
we just talked about, Seals, matter, SMOTE to create a
roughly balanced data set. Wait, so they use the old tech
to prep the data for the new tech?
Why do that? Think of it as priming the
canvas before you paint. If you feed a drastically
imbalanced data set like 95 to 5 into again, even a conditional
Jan might struggle to converge at the beginning.
It's a hard problem. OK by balancing it roughly with
eight is 1 first maybe to like a 6040 split?
They gave the deep CT Yan a much easier starting point, a head
start. That makes sense.
And then Step 3, advanced augmentation.
Right. They feed this primed, roughly
balanced data into the deep CT Jan but they modified the
architecture. This is the core innovation.
I think they added a resnet backbone.
This is the most technical part of the paper and probably the
most important. Resnet stands for residual
networks. We usually hear about this in
the context of image recognition.
Resnet 50, Resnet 101. So why are we jamming an image
recognition architecture into a generator for tabular data?
It seems like a weird. Fit It's all about solving a
classic deep learning problem called the vanishing gradient.
OK, Eli, five time. Give me the simple version.
What is the vanishing gradient? Imagine you're playing a game of
telephone, but the line of people is a mile long.
That's a deep neural network with many layers.
OK, you whisper a message at the start.
That's your input data. By the time it gets to the
person at the very end, the output layer, the message is so
distorted and quiet and basically gibberish.
The person at the end has no idea what the original signal
was. The signal just dies out as it
passes through all the layers, correct?
And in a neural network, that signal is the gradient.
It's the error signal that tells the network how to update its
weights, how to learn. If that signal dies, the model
can't learn. It stops updating.
It gets stuck. And resnet fixes this with skip
connections. The shortcut.
That's the perfect way to think of it.
Instead of passing the message only to the very next person in
line, you also get to shout it to the person two or three spots
down the line. So the original information gets
to jump ahead. Mathematically, instead of a
layer trying to learn the full complex function FX, it only has
to learn the residual or the difference FX plus X.
That plus X is the skip connection.
It carries the original unmodified input forward.
So it never forgets where it started, it always has the
original context. Precisely.
And for tabular data this is absolutely be crucial because
the correlations can be very subtle and exist at different
levels. You have simple patterns like
older people have higher risk, but also very complex high level
patterns like this specific enzyme level combined with this
blood type and this genetic marker equals danger.
And the skip connections help the model capture both.
Exactly. It allows the generator to
capture both the simple low level features and the complex
high level dependencies without getting lost in the noise of a
very deep network. It just stabilizes the whole
training process. OK, I want to get granular on
the hyper parameters here. The paper actually lists the
settings they used. So if I'm a data scientist
sitting at my GPU right now trying to reproduce this for a
project, what am I typing into my code?
Get your notebooks out. Yeah, for the core deep CT Jan
training, they kept the learning rate very, very low .0002.
That is tiny. The default in most frameworks
is what .0001? Why go so slow?
Stability. GNS are notoriously unstable.
It's a delicate tug of war. If one side, the generator or
the discriminator, learns too fast, it overpowers the other
and the whole system falls apart.
The rope snaps. The model just oscillates and
never finds a good solution. Right.
So you have to creep up on the solution.
Slow and steady adversarial training.
Makes sense. What about batch size?
128 That's a pretty classic Goldilocks number.
It's small enough that it fits in memory and provides a bit of
stochastic noise, which helps with generalization, but it's
also large enough to give you a stable, reliable estimate of the
gradient. And the embedding dimension for
a categorical features. We use between 128 and 256 and
they said it depends on the complexity of the data set.
This is for converting those text based categories like city
or hospital ID into dense numerical vectors that the math
can actually work with. Got it.
Now for the Resonant backbone specifically, what did they use
there? They used a Resonant 18
architecture. That's one of the lighter,
faster versions of Resonant. But the real secret sauce I
think was the regularization they applied.
They use dropout at a rate of .3.
Which means they are randomly turning off 30% of the neurons
during each training step. Yes, exactly.
And they also use weight decay at 1 E 5.
Both of these are absolutely crucial because remember, our
starting data set of sick patients is tiny, maybe 50 or 60
people. Deep learning models are
effectively massive memory machines.
If you let them, they will just memorize those 50 patients
perfectly. That's overfitting.
Extreme overfitting. The model becomes a parrot, not
a thinker. Dropout forces it to be robust.
You can't rely on any single neuron, because that neuron
might be switched off at any time.
It forces the model to learn the underlying concept of the
disease, not just the specific patient records in the training
set. OK.
So we've built the generator, we've primed it with ATIS YN,
we've stabilized it with Resnet and we've tuned it with these
very specific slow hyper parameters.
We generate our new synthetic data.
Now we have a nice big balanced data set.
We still need a classifier to actually diagnose the patients.
Did they just use something simple like random forest?
They compared a few. They looked at Random Forest,
XG, Boost, KNN, but the hands down winner, the one they
focused on was Tabnet. Tabnet, another deep learning
model. Why use a deep learning
sledgehammer when a screwdriver like Random Forest usually works
just fine for tabular data? Because Tabnet is a deep
learning model that's specifically designed to beat
decision trees at their own game, it has an architecture
that really tries to mimic the way a decision tree, or even a
human doctor thinks. It uses a mechanism called
sequential attention. Sequential attention.
What does that mean in this context?
Think about how you diagnose a car problem.
You don't check everything all at once.
You pop the hood, You check the battery first.
Is it dead? No.
OK, now check the starter. Is that making a clicking sound?
It's a step by step process of elimination.
Exactly. Tabinet does this.
It uses a learnable mask at each step to focus on a specific
subset of features. For the first step, it might
only look at age and blood pressure.
Then based on that, for the second step, it decides to look
at hemoglobin and leukocytes. So it learns not just what's
important, but in what order to look at things.
And it learns what to ignore this.
The other key part uses a very specific activation function
called antmax. Antmax.
I've heard of softmax. Softmax is what's typically
used. It gives a probability score to
everything, but nothing is ever truly 0A feature might be
.00001% important at Max, allows for true sparsity.
It can actually say you know what, these 10 columns in the
spreadsheet. They're completely irrelevant
for this decision probability. Zero.
It effectively deletes the noise from the calculation.
Which is huge. It makes the model much more
efficient and critically, more interpretable.
It tells you exactly which features it chose to ignore.
So that's the full stack, a two element baiting feeding into a
deep CTGN with a resnet backbone which generates data to train a
tab net classifier. That's the recipe.
Now the big question, does it actually work?
Or is this just a cool engineering science project that
looks good on paper? The validation results are
honestly startling, but first we have to define how they tested
it. This is a hell I will die on.
You have to use TSTR. Trade on synthetic, test on
real. It is the golden rule of
synthetic data validation. You train your model only on the
augmented or fully synthetic data, but you must must must
test it on a holdout set of real flesh and blood patients that
the model has never seen before. Because if you test on your own
synthetic data, you're just grading your own homework.
Exactly. It's a meaningless self
congratulatory loop, but if a model that was trained entirely
on fake data can accurately diagnose a real patient it has
never met. And you know, the synthetic data
successfully captured the fundamental underlying rules of
the disease. That's the proof.
So what were the scores? What did they find?
OK. For the chronic kidney disease
data set using the full hybrid framework, they achieved 99.4%
accuracy. Wow, that is that's basically
perfect. What about the others?
For COVID-19 they hit 99.2% and for Ding Fever 99.5%.
That's incredible. How did the more traditional
models compare on the same tasks?
They were good, don't get me wrong, but Tabnet consistently
beat Random Forest and XG boost across the board on these
augmented data sets. But the most interesting failure
to me was KNN. Key nearest neighbors, the one
that works based on distance. Right, it's a very simple
algorithm and in some of their experiments KNN actually
performed worse when they added the synthetic data.
How is that possible? How does more data make a model
worse? Because KNN relies purely on
geometric distance and synthetic data, even very high quality
data like this introduces a tiny, tiny bit of noise.
It's not perfect, it fuzzes the decision boundaries just
slightly. And KNN is super sensitive to
that. Very sensitive.
It got confused by these new fake neighbors that weren't
placed perfectly in the geometric space.
It started making mistakes it wouldn't have made before.
That is a huge practical take away for anyone listening.
If you are going to implement this kind of synthetic strategy,
you might need to upgrade your classifier too.
You can't just plug this into a legacy KNN system and expect
magic to happen. You really can't.
You need a more robust classifier like tab net that can
use an attention mechanism to filter out that slight noise and
find the real signal. There was one other metric in
the paper I wanted to ask about, the similarity score.
They said they achieved between 84 and 87% right?
Which sounds like AB Plus why wouldn't we want 100%
similarity? That's a great question because
it's counterintuitive. If your synthetic data is 100%
similar to your real data, you've just cloned it.
You've perfectly memorized the training set.
Which is a privacy violation and it's terrible for
generalization. Exactly.
You haven't learned anything new.
You want the synthetic data to be close enough to capture the
statistical properties the physics of the system, but
different enough to be novel and to cover new ground. 87% is
actually a very healthy score. Means you're in the sweet spot.
So the model is incredibly accurate, but in medicine,
accuracy isn't enough. I can't just walk into a
patient's room and say, well, the computer says you're sick,
so bye. I mean to know why?
The black box problem. Yes.
This brings us to the fifth section of the paper,
Interpretability. To open up that black box, they
used SHAP values. SHAP Shapley additive
explanations. It comes from game theory,
doesn't it? It does, yeah.
It basically treats the model's prediction like a cooperative
game where every feature, every symptom, every lab result is a
player on a team. OK?
And it calculates how much each individual player contributed to
the final score, to the final diagnosis.
It tells you who the MVP was for that prediction.
So it gives us a clinical reality check.
The ultimate reality check If your model is 99% accurate, but
it tells you the most important feature for diagnosing cancer is
the patient ID number, you know something is fundamentally
broken. You know it's garbage.
So let's look at what the model actually learned for the
COVID-19 data set. What was the top predictor
according to SHAP? The top feature was leukocytes.
White blood cells. Exactly.
A high white blood cell count indicates an infection.
It's one of the most basic tenets of medicine.
So that's biology one O 1 the model learned the right.
Thing it validates that the synthetic data preserved the
true underlying biological signal for kidney disease.
The top 2 features it flagged were hemoglobin and hematocrit.
OK, so low hemoglobin means anemia, and kidneys produce a
hormone erythropoietin that tells the body to make red blood
cells. So if your kidneys fail, you
can't make that hormone and you become anemic.
See, the model figured out the entire renal anemia connection
just by looking at the synthetic data.
It learned real medicine. But the dengue data set, that
one had a really interesting curveball.
Oh. What was it?
The top predictor wasn't a blood test.
It wasn't a symptom. It was the column labeled year
the. Year like the calendar year the
patient was seen 2018-2019. And at first you might think,
OK, that's a data leak, that's a bias, that's a mistake.
But then you should think about what dengue fever is.
It's an outbreak disease. It's carried by mosquitoes.
It's epidemiological, it comes in waves.
It's seasonal encyclical. The model learned that if a
patient walks into the clinic during 2019, which is a big
outbreak year, the data set their prior probability of
having dengue is way, way higher than if they walked in during
2018. That is fascinating.
It didn't just learn Physiology from the synthetic data, it
learned epidemiology. It learned about public health
trends. And it also found that the
remarks column, which was this unstructured text notes for the
doctors, was highly predictive. Which Tabnet is well suited to
handle because of the embedding layers we talked about.
It shows that the deep CTGN was able to encode these really
complex nonlinear relationships between calendar year, the free
form text notes and the final diagnosis.
It's incredible, really. So we have a working system,
it's accurate, it's interpretable, but I want to
bring this back down to earth. This is the practical AI digest.
There is no free lunch in engineering.
What is the cost? What's the trade off for this
fancy hybrid framework? The cost is time and compute and
the paper is very transparent about this which is great to
see. They timed it.
Training a standard Smoke plus a Tabnet model took them less than
10 minutes. So, quick coffee break and
you're done, right? But training the full hybrid
model, the ADCNN pre balancing, the deep CDD GAN with a resnet
backbone and then the Tabnet classifier, that took 42
minutes. So that's more than a four fold
increase in training time. It is now.
In the grand scheme of things, 42 minutes isn't long if you're,
say, training ChatGPT from scratch.
But in a clinical production environment where you might be
retraining these models every night on new data, that adds up.
So you are paying in GPU hours for this extra couple of
percentage points of accuracy. You are, and you have to ask if
it's worth it. For life threatening diseases
like these, I would argue absolutely yes, that extra 1 or
2% in accuracy translates to real people not getting
misdiagnosed. But if you're using this to
predict, say, customer churn, maybe the simpler, faster SOD is
good enough. The Blue Gen.
AI source had some great implementation advice here too.
They talked about the point of diminishing returns.
It's. So crucial for the builders out
there. Don't just turn on the generator
and create a billion new rows of data.
It doesn't work that way. Blue Gen. suggests starting by
generating 2 to five times your original minority data set size.
I have 1000 real rows of the rare disease I should aim to
generate, maybe 2000 to 5000 synthetic ones.
Exactly. The study showed that if you go
much beyond that, you aren't really adding new information
anymore. You're just adding noise and
increasing your training time for no benefit.
The accuracy starts to plateau. You have to find that sweet
spot. And what about their hybrid
workflow idea? This is just best practice for
safety and reliability. Use the synthetic data for
training. Use it to cover all those weird
edge cases, the Leicester City moments we talked about, but
never let synthetic data leak into your final validation set.
Your final exam must be on real world problems.
Always, and you have to constantly monitor for
hallucinations. If your similarity score starts
to drop or your TFTR performance tanks, it might mean your
generator has drifted and is inventing relationships that
don't exist in reality. You need those automated checks
before you deploy anything to a real clinic.
So to summarize the entire stack we've uncovered today, we have a
fundamental problem. Imbalance causes lazy AI that
misses the most critical cases. We have a solution, the hybrid
framework. We prep the data with a classic
tool like Adis Wyan. We generate new data with a
powerful conditional deep CT jam that's been stabilized by Resnet
skip connections, and then we classify with a smart, attentive
model like Tabnet. And we validate everything with
TSTR. It's a robust, modern and
frankly very effective pipeline. As we wrap up, I want to leave
our listeners with the thought that came from the conclusion of
the paper. They mentioned real time
streams. This felt like a genuine glimpse
since the near future. This is where it gets really
exciting. Beyond just a single static data
set. And imagine a synthetic data
generator that isn't a script you run once a year.
Imagine it's a live running service.
So it's connected directly to the hospital's intake feed.
Exactly. Every single night it
automatically ingests the day's new patient cases.
And if a new strain of a virus starts to appear, like we all
saw happen with the COVID variance, or if the common
symptoms of Denguing start to shift slightly because of a
climate change factor. The generator would see that new
pattern emerging in the real time data.
And it would immediately generate thousands of new
simulated cases of this new pattern.
It would then use that data to retrain the main diagnostic
model all overnight automatically.
So by the time the doctors clock in for the morning shift, the AI
has already practiced on thousands of cases places of the
new strain before the human doctors have even seen their
second real patient of the day. Precisely.
It turns the AI from a historian, something that only
looks at the past, into a scout. It can adapt to a changing
disease faster than a human run public health system can.
That is slightly terrifying, but also incredibly hopeful.
It's the future of adaptive real time diagnostics is where this
is all heading. So here is your call to action
for the week. If you're an engineer sitting on
an imbalance data set, and I know you are because basically
all real world data sets are, try implementing Deepctgen.
Don't just settle for the old SEM OT standby.
But please don't forget the resnet skip connections or your
gradients will vanish into thin air and you'll spend a week
debugging. It and always always check your
feature importance if patient ID is your top predictor.
Delete your model, delete your features and start over from
scratch. Wise words.
Thanks for listening to this deep dive.
We'll catch you on the next one. Goodbye and Abby Building.