Hi, Alex from OverOps here. We've created the "Bullshifier" internally to stress test edge cases with our own monitoring tool (http://www.overops.com).
Whenever an exception or a logged error / warning happens in production, it shows you the complete source code leading to it, and the full variable state across the entire call stack.
Using Java Bullshifier, we're able to generate random applications with call stacks that can go thousands of frames deep and other quirky behaviors. This helps us make sure the overhead is unnoticed, even in the most extreme cases.
I don't know why, but the fact that this is an actual tool that's been designed to solve a real problem that you guys faced (as opposed to being a random project that somebody dreamed up to make fun of Java or to see if they could do it) makes me really happy.
Java is alive and kicking. It's the most popular programming language in the world. As such, it also attracts most of the fire. We hope to see more colorful projects like these go open source. Thanks for the comment! :)
I've been wanting to build a generic version of this type of tool for quite some time.
When I'm waiting several minutes for our app server to startup, I often ponder what life would be like if we had chosen language X or another stack for our application.
I've primarily worked on JRE-based stacks for some time and once a project reaches a certain size developer happiness begins to decline due to large compilation times, app startup times, etc. Even incremental compiling or hot-reloading of classes seems unreliable at a certain app size (either due to memory or some other reason).
It would be neat if there were a tool to generate large and complex apps with various stacks to get a feeling what the developer experience would be like 1-2 years down the road once the app reaches a critical mass.
That is a good idea for testing out new stacks for that shiny new rewrite. One thing I've realized is most projects are not designed for 100+ developers working on it at +400kloc of code.
We adopted https://buckbuild.com/ at work for our android builds, and it has sped it up a lot compared to our previous gradle build process.
We have a self-hosted Jira/Confluence server running on Windows that's reached end of life....I can relate to this. Fortunately no Oracle in the mix other than Java.
I did once work on a project that had a class FluxCapacitor. It was used in testing to set the time. It worked, but had about as many zany unexpected consequences as its fictional namesake.
Hey, at the moment we're only interested in JVM languages, so Java was the best fit for this project. We'll be adding .NET support for OverOps in the future so that might be something we'll be looking at for Bullshifier as well.
Basically, the tool is built with Groovy (many projects use it as a scripting language for Java), and there are 4 sections to each randomly generated class:
1. Variable definition, with random types, random names, and values.
2. A section that throws exceptions, or prints to the log, according to the config settings for the rate of events. This also includes a “suicide” function that stops the execution of the program.
3. Calls the next method.
4. Random code that never runs.
There are some advanced config settings described in the Github README.md
At first I thought this another one of the several Java shaming projects that have been posted in the past on HN (usually in regards to the ridiculous class names of some projects ... aka Spring).
But this project actually has some serious utility particularly for byte code manipulators are just alternative compilers like AspectJ. Particularly AspectJ as I have seen it fail on things that neither the Eclipse compiler (JDT) or Open JDK fail on.
I'm glad to learn that entire businesses exist thanks to the fact that Java has overly verbose exception outputs which are designed to hide the information you are seeking between 200 lines of call stack prints.
As an experienced Java developer I love long Java stacktraces and I miss them in most languages. In 90% cases stacktrace allows to find a bug without any debugging, because it's just very obvious where it is.
Of course if someone doesn't care about exceptions, it's just a bad style. Exception in the production log is like red light and alarm sound, BUG-BUG-BUG. I saw projects, which throw stacktrace after stacktrace, gigabytes of stacktraces (and then archived those teralogs and noone ever watched them). But those projects have much more problems, verbose logs is the least of them.
Yes, with heavy framework usage, especially when those frameworks generate proxy classes and wrappers and god knows what, it's often 2-3 lines of useful information between 90 lines of not-so-useful library or even autogenerated code. But only developer can judge it, environment should preserve anything.
The problem has historically been that most J2EE frameworks overly rely on delegation, because of GoF.
I don't like seeing the same 5 stack frames in the same class appear three separate times in a single stack trace. Because what does that stack trace say?
It says it took Foo five function calls -on itself- to figure out it's not responsible for an operation. So Foo asks Bar to do it, and after half a dozen more function calls someone asks Foo all over again. Who takes 5 function calls to figure out it's STILL not responsible for this action. Lather, Rinse, Repeat.
And how do you set a breakpoint in that call tree? Which time do you want to debug it? How many more times was it called before the exception was thrown?
Your concern is right, but how could we have solved it? I think it's a composition of the Reflection API, the Java style of not accepting optional function parameters, and the pluggable systems.
We call the first framework function, which calls the second function with the default values for the arguments that were optional. The second function looks up the application's method using Reflection. Because Reflection code is verbose and has a lot of tricky cases, we want to share code between all places where we call it, so it delegates calling the API in 4 different methods.
Then come up the ServletFilters in an HTTP application, which is an example of pluggable system. If we declare 10 filters, they all do something like check the authentication, gzip the response, check the XSRF token, transform a WebApplicationException into a 404/500 reaponse. Unfortunately they all appear as "ServletFilters" in the stacktrace, until they delegate to the final HTTPServlet itself. Another example of pluggable system is OSGi: Each method call is wrapped in 5 function calls if the method is in another classloader.
There's only one improvement I'm able to imagine: Java could define a pluggable system at the language level, so we don't have to implement them using function calls. This pluggable system stretches for all delegation needs, from OSGi to ServletFilters to apps where the list of delegates were defined by a REST api, etc. Sounds like a big JSR...
Barbara Liskov and Bertrand Meyer both have talked about separating decision making from execution. And though there work predates widespread adoption, I find this happens to work remarkably well for unit testing as well.
The thing to remember is that when Java was young, the pool of people who grasped OOAD was very small. Nearly all of us were making it up as we went along. So Design Patterns came out and everyone glommed onto that book, and then went out and wrote code so awful and convoluted that it's now a joke.
J2EE was built up in that space, on a foundation of bad assumptions and unfounded theories. I was simply amazed that the same company that brought us the Eight Fallacies of Distributed Computing brought us the J2EE 1.0 spec. I was so appalled at its quality that I switched to UI development until well after the 2.0 spec was out and implemented.
It says something profound that every vendor that had any succes with those versions of the spec did it by defying the spec and supporting local dispatch. A lot of people couldn't scale their app past two machines, but load balancer eventually came around and fixed that.
Anyway, my point is you have a bunch of APIs written by newly minted experts, many of the foundational ones by people with absolutely no awareness of the physics of distributed computing, it's going to take a lot to spackle to cover those sins. Some did, others doubled down, but overall every problem was solved by another layer of indirection.
When refactoring came to the fore I hoped the industry would hastily reverse course. Spring was supposed to fix a lot of this but from my view, Spring became the thing is sought to replace. Full of cryptic levers and dials and massive indirection.
I think it was Mike Feathers who attributed this to cowardice. People afraid to make decisions put all the decisions behind a layer or two of indirection so they can "cheaply" change their minds later. But again we are back to Meyer and Liskov here.
If A then B (where A is the decision and B the action) allows you to change A without adding much if anything to B, or add to B without changing A. If this shows up in an inheritance hierarchy then you need about half as many overridden methods for the same call tree. Which means a shallower call tree, and code you can speak out loud.
This, and also the fact that once you've been maintaining an application for a short period of time, you quickly learn to recognize those "framework classes that are always there" in the stack trace and skip over them quickly. They are not really a burden, and as long as you're viewing the stack trace in an editor that responds to your mouse's scroll wheel it's no big deal.
I will agree with vbezhenar that Java stack traces are extremely useful. When something has gone wrong with an application in production, it is useful to know the call stack that led to the problem. I know that I didn't just fail in the method `Foo.doFoo()`, I failed in that method which was called by `Baz.doBaz()`. I know these two pieces of code are interacting which is a big clue as to the problem.
vbezhenar also offers good advice that logs of production software should be empty of exceptions and stack traces -- that the presence of anything is the sign of a problem, and thus rare in well-maintained software. Honestly, well-operating production software shouldn't emit any logs at all. Metrics and activity records if you need them, sure, but there's no need for logs. Logs can consequently be used to quickly pinpoint unexpected things happening.
agree on mostly everything apart from last part - even well-working app should have some debug/info/warn statements. ie my current app - messaging integration tool build on apache camel, I log rather more than necessary, as messages go through various components, operations made on them etc. Each component isn't aware of others, so logging current state only if exception happens will lose some good info on overall situation happening.
a bit too verbose, but the amount of info I can get from UAT/PROD envs in case of an issue is mostly enough to fix the bug. verbosity can be solved easily by proper rolling & backup of logs, that's not rocket science.
and then archived those teralogs and noone ever watched them
I champion Sentry wherever I go, as it has an outstanding "fold these common explosions into one event" behavior, in addition to managing the _lifecycle_ of an exception/error, from first encounter through resolved in release, to the dreaded reoccurence.
I do keep a terminal open for my curiosity but I'm much happier with computers watching logs than me having to
I think the tool that we're building (http://www.overops.com) could be something that might help with the log data overload. It basically gives you everything you need to troubleshoot an error - without relying on log files, while deduping recurrences of the same event.
It gets all the required information (source, stack and variable state), through a native JVM agent that transmits everything directly to the tool.
The level of verbosity is good, however when you replace almost all your catch exceptions with ex.printStacktraces() it gets enormously large and unmaintainable across a large project when too many things prints stack taces concurrently. Not the way I like to do it, however too many people do that unfortunetaly: http://plg.uwaterloo.ca/~migod/846/current/projects/09-Naksh...
I prefer something in between. Something like: the method that called the current method, the line number and a sentence describing the exception. No need to know the function-that-called-the-function-that-called-the-fun[...]
In my experience the direct caller usually tells me very little about the root cause of the exception. The interesting context is often at least ten frames separate from the proximate cause in even moderately complex Java programs. Of course, often there's an additional couple dozen lines in the trace that aren't very useful...
That's sounds like you work on very small projects. That function-that-called-the-function-that-called-the-fun could behave very differently through that path due to calling parameters, environment or data issues.
Sentry doesn't give you the local variable state at the moment of error. Only environment related data. We use it as well, but only for JS frontend. It's more of an error tracker, rather than a root cause analysis tool. The benefit is that it's wider and doesn't only focus on Java, but you'll still need to spend time going through logs to actually troubleshoot those errors and understand what was wrong with the state that caused them.
With OverOps, you go deep on Java, with every variable value, overlaid on the code across the call stack. Without looking at / relying on logs.
There's a short video here (http://www.overops.com/) and you can see some more samples after you sign up.
Hey, Lewis from Sentry here. "Sentry doesn't give you local variable state" is not entirely accurate - it varies from platform to platform. We don't do it for Java (you're obviously ahead of us there), but we do with at least Python and PHP off the top of my head. I wish we could do it for JavaScript too, but it's only partially feasible and only in a very questionable way.
Note that in your browser JS use case, we capture breadcrumbs to show you console logs, user interaction events, XHRs, navigation events, prior errors, etc leading up to the error, so you should be able to easily just reproduce the error state rather than digging through logs to find out what it was. If you still find yourself needing to dig through logs to troubleshoot JS errors reported via Sentry, I'd really like to hear more about your experience to see if we can make it better.
Java stack traces are verbose because Java devs write verbose code. Java stack traces are nice and readable. Just look at a Go stack trace, it'll make you want to vomit.
Well placed print statements. I print only a few things. Of course, in Java & eclipse, it is hard to see the result because so much stuff gets printed (The stack trace, Maven showing useless details, tomcat doing the same, etc.).
In C/C++/rust/JavaScript/Python, the last things you see on a failed build/run is usually the detail that will help you fix the problem. The information is organized to be read by someone.
You can easily control logging level with slf4j etc, you can run maven in quiet mode if you want to see only errors. I don't think these are java lang issues, its more of a people problem that they don't quite understand the language and the tools they use in depth.
I've done my fair share of work in those languages and it almost never has anything to do with the last line. Typically following back into some stupid === vs. == error or some dynamic typing issue many frames up the call stack. Even C++ is a nightmare to traceback when you use boost/STL heavily
If you use the right random word combinations for the class name generation, and the code actually does something, this may be near-indistinguishable from a real Java codebase.
I wonder how much I'd have to spend on AWS if I just wrote the acceptance tests for our next project, then ran this in a loop/cluster until it wrote something that passed UAT... :)
That would actually work [1] to certain extend but once you have to prove your real skills during a technical interview it will be clear that you are not a real Java developer unless, of course, you are. If you are a Java developer this tool will (probably?) have the effect that you suggest, to increase the probability of an ATS [2] to hit your online developer persona.
I will actually try this because as people have mentioned before hiring is broken and this is either a way to exploit and/or bypass some of the inconveniences like coding challenges and the like used to filter non-programmers.
It was about time somebody creates something like this. I do have a complain, classes are named random. If you used a tool that generates human like names, it would still be random but looked much better.
I will see to create a similar rails tool. I intend to test just stress of people who will look at the code and try to understand what it does.
If you have the time (probably not :) ), there was a really cool project that took names from GitHub and generated random names using Markov Chains... maybe use the same for the class names :) .
Neat idea! As a mostly-Java coder (by choice), I love both the approach to testing your own tool as well as the slight side blow into the Java community regarding long stack traces full of framework-contributed delegation classes and virtual proxies.
But...not more than 20k classes? That doesn't really count as "massive Java application" these days, more like an average-sized enterprise app when including all its dependencies. You would at least triple that to reach "massive" scale ;-)
Kind of off topic, but this reminds me of my internship this summer. What sucks about Spring is the horrible error messages. If there is something wrong in the XML configuration, or dependency configuration, it doesn't actually tell you that. Instead, it shows a deep stack trace with some strange exception that doesn't relate to the problem at all.
Actually this could be a good platform for generating licensing algorithms that would be included in commercial software. This level of obfuscation would really be hard to crack by reverse engineers that produce cracks/keygens.
Whenever an exception or a logged error / warning happens in production, it shows you the complete source code leading to it, and the full variable state across the entire call stack.
Using Java Bullshifier, we're able to generate random applications with call stacks that can go thousands of frames deep and other quirky behaviors. This helps us make sure the overhead is unnoticed, even in the most extreme cases.
Would be happy to answer any questions.