> improvements that haven't been published yet to make it work in this domain
Are you able to share what some of those are?
> You can choose what parameters to fix and which to keep variable over the training process
Can you explain more about the parametrization? Do your parameters correspond 1-to-1 with schematic parameters like transistor sizes or resistance/capaciance values? Or internal transistor model parameters? Or are they more abstract mathematical parameters?
> I'm not sure I quite understand the distinction that you're drawing and measuring errors in a sensible way here is actually somewhat non-trivial.
Circuit simulator vendors often market their accuracy in terms of "% SPICE Accuracy", and what they mean is if you run a simulation and measure some parameters (usually something like RMS noise voltage or signal-to-noise ratio), then those measurement results will be within 1% of what the measurement results you'd get from running the same simulation with full-accuracy SPICE.
The other way of measuring simulator accuracy is in terms of dynamic range. For example if I have a noisy sine generator circuit where the signal has rms amplitude 1, and the noise is 1e-4 rms, I need to make sure the numerical noise of the simulator is much less than 1e-4.
The first is sort of relevant in your case as a comparison between surrogate and full simulation. The second is an absolute measurement of a single simulator's (or surrogate's) accuracy.
> I believe the error rates Chris quoted are from a smooth distance metric between the signal generated by a full simulation and those generated by the surrogate.
This is interesting because it makes sense as an application-independent metric for measuring the accuracy of your surrogate. It's not immediately clear to me how this would translate to circuit performance accuracy in all cases. However, in one specific case - a Digital to Analog Converter - that 1% smooth distance error could be catastrophic for some performance metrics depending on how it behaves.
That might be an interesting thing for you to consider investigating through the course of your research.
> We are expecting speedups on real-world problems just by using this framework for baseline simulation
Very cool, and I'm looking forward to seeing some data there.
One last question - why did you choose this particular DARPA project? Was it any specific interest/relationship with the circuit design industry? Or did it just happen to be a very cool application of CTESN?
The papers are being written, so should be public in a few months, but I can't go into detail quite yet.
> Can you explain more about the parametrization? Do your parameters correspond 1-to-1 with schematic parameters like transistor sizes or resistance/capaciance values? Or internal transistor model parameters? Or are they more abstract mathematical parameters?
All of the above. We have a fully symbolic representation of the circuit, so any part of it can be replaced by a parameter. Transistor sizes and device values are a natural thing to do, but the system is generic.
> % SPICE Accuracy"
Heh, we've actually found some SPICE implementations to have significant numerical issues when compared to our baseline simulator, which has some fancier integrators - I suppose that would make us worse on this metric ;).
> That might be an interesting thing for you to consider investigating through the course of your research.
Yup, characterizing error trade offs is one of the primary outcomes of this research.
> One last question - why did you choose this particular DARPA project? Was it any specific interest/relationship with the circuit design industry? Or did it just happen to be a very cool application of CTESN?
Circuit design was my first job, and I have a second desk with a soldering iron and test equipment, though it's collecting dust ;). I have a bit of a hypothesis that - between open source tooling becoming more mature, and more people getting into the chip design space for custom accelerators - we're on the cusp of a major overhaul in EDA tooling. I thought we'd be good at it, so I was advocating internally for us to start up a team in the domain. We have a bit of an "if you can get it funded it, you can do it" attitude to those sorts of things, so I was trying to find the money to jumpstart it, and this is that.
> Heh, we've actually found some SPICE implementations to have significant numerical issues when compared to our baseline simulator, which has some fancier integrators - I suppose that would make us worse on this metric ;)
Hah. That's not surprising. Our vendors generally mean spectre when they talk about full accuracy, but I have a laundry list of grievances when it comes to spectre and its accuracy settings... What are you using as your baseline?
> Circuit design was my first job
Glad to hear. This industry needs a lot of modernization.
> I have a bit of a hypothesis that - between open source tooling becoming more mature, and more people getting into the chip design space for custom accelerators - we're on the cusp of a major overhaul in EDA tooling
I hope you're right. The current state of circuit EDA tooling is abysmal. I think on the analog/RF side, the ecosystem is so ancient and entrenched that it will take a herculean effort to make any real strides, especially because the golden age of analog/RF semi startups is over. But digital design is very much becoming open source, as you mentioned, and maybe that will eventually bleed over.
I wish all types of circuit design were more accessible to the world - lots of emerging economies could use it, but the up front cost is just so high. There's been a massive surge of software dev recently in the Middle East and Africa, and hopefully hardware dev follows suit.
Are you able to share what some of those are?
> You can choose what parameters to fix and which to keep variable over the training process
Can you explain more about the parametrization? Do your parameters correspond 1-to-1 with schematic parameters like transistor sizes or resistance/capaciance values? Or internal transistor model parameters? Or are they more abstract mathematical parameters?
> I'm not sure I quite understand the distinction that you're drawing and measuring errors in a sensible way here is actually somewhat non-trivial.
Circuit simulator vendors often market their accuracy in terms of "% SPICE Accuracy", and what they mean is if you run a simulation and measure some parameters (usually something like RMS noise voltage or signal-to-noise ratio), then those measurement results will be within 1% of what the measurement results you'd get from running the same simulation with full-accuracy SPICE.
The other way of measuring simulator accuracy is in terms of dynamic range. For example if I have a noisy sine generator circuit where the signal has rms amplitude 1, and the noise is 1e-4 rms, I need to make sure the numerical noise of the simulator is much less than 1e-4.
The first is sort of relevant in your case as a comparison between surrogate and full simulation. The second is an absolute measurement of a single simulator's (or surrogate's) accuracy.
> I believe the error rates Chris quoted are from a smooth distance metric between the signal generated by a full simulation and those generated by the surrogate.
This is interesting because it makes sense as an application-independent metric for measuring the accuracy of your surrogate. It's not immediately clear to me how this would translate to circuit performance accuracy in all cases. However, in one specific case - a Digital to Analog Converter - that 1% smooth distance error could be catastrophic for some performance metrics depending on how it behaves.
That might be an interesting thing for you to consider investigating through the course of your research.
> We are expecting speedups on real-world problems just by using this framework for baseline simulation
Very cool, and I'm looking forward to seeing some data there.
One last question - why did you choose this particular DARPA project? Was it any specific interest/relationship with the circuit design industry? Or did it just happen to be a very cool application of CTESN?