If you are saying that it’s better to run native than Terraform then to me it means that Terraform is not good enough - which is also my experience from poking it. It covers all use cases on all clouds, buy only ~80% of each use case. The rest you have to figure out by yourself. Which often means diving into CloudFormation and such - so basically losing the advantage of Terraform.
Basically right. And every time the platform changes, terraform has to implement it too! And more often than not, it's supported in the providers implementation than terraform.
I think terraform was better before, but now CloudFormation for me is good enough, so I don't see the point of Terraform.
Cloudformation also has Drift detection now and I really dislike Terraforms state files.
This is only true recently, and largely only because the CF team got sufficiently embarrassed by TF have support for things they didn’t literally months ahead of time.
I find it easier to read Terraform over CloudFormation. I also find it easier to write.
Anecdotally: non-devops developers find it easier to edit Terraform than CloudFormation. They understand the resource and data blocks much easier than YAML.
Terraform bridges the gap between purely YAML CloudFormation and purely JavaScript API calls. (JavaScript used as an example there, it could be any language).
Terraform makes tradeoffs to satisfy its problem domain and be easier to use than CloudFormation. Plus Terraform often gets features faster than CloudFormation because cf seems to lag behind the AWS API.
I'd like to write some of my simpler Terraform stuff in AWS CDK, Pulumi, or something similar, to see if I can make use of more language features like inheritance or better decision logic. (currently I try to keep as much logic outside of Terraform as possible. It's supposed to be declarative and not have "real" coding language features shoved into it)
For one, it's closer to a proper programming language as opposed to straight up data interchange format. Sure if you write it in YAML than you can take advantage of variables but YAML's syntax for variables is pretty gross.
Comparing CloudFormation to Terraform is a little like comparing HTML and CSS to Javascript (though Terraform isn't nearly as nice to code in as Javascript -- and I'm not exactly a big fan of Javascript). You can cover most use cases with plain HTML and CSS but the moment you need to get a little more intelligent with your code you get stuck.
> For one, it's closer to a proper programming language as opposed to straight up data interchange format. Sure if you write it in YAML than you can take advantage of variables but YAML's syntax for variables is pretty gross.
So what is stopping you from using "a proper programming language" to generate the json/yaml cloudformation template?
This is what you see in GCP docs from day one. On AWS, they brainwashed everyone in this corner of "static template with parameters", so that you can "reuse" a template to build your custom stack. It's great for "look what I can do, mom" (but I have no idea what it's doing) but nobody sane would ever trust a 1-km long yaml/json and deploy it. So if you anyway have to inspect it, why not make it easy to inspect? Split into modules, add docs, etc = code to run.
I have no idea how we switched from random scripts to "reusable" random scripts (ansible &co) to random static configuration and then the cherry on top: "reusable" random static configuration. Insane. Abstractions on top of abstractions.
CDK is on the right track, but even there it's a mess, again for the sake of hiding complexity: constructs and deployment. Where did One thing well and Keep it simple stupid go? :)
> So what is stopping you from using "a proper programming language" to generate the json/yaml cloudformation template?
Nothing and there are already products out there that offer that. However I think the issue will always fall back to the problem that you're compiling from a functional or procedural language down to a dumb data interchange format. That can cause a variety of issues such as losing your carefully crafted order of execution.
> This is what you see in GCP docs from day one. On AWS, they brainwashed everyone in this corner of "static template with parameters", so that you can "reuse" a template to build your custom stack. It's great for "look what I can do, mom" (but I have no idea what it's doing) but nobody sane would ever trust a 1-km long yaml/json and deploy it. So if you anyway have to inspect it, why not make it easy to inspect? Split into modules, add docs, etc = code to run.
I don't think anyone was "brainwashed" by CloudFormation and the solution you describe is exactly the approach Terraform takes.
> I have no idea how we switched from random scripts to "reusable" random scripts (ansible &co) to random static configuration and then the cherry on top: "reusable" random static configuration. Insane. Abstractions on top of abstractions.
This I agree with. It's not just AWS though, you see YAML-based config all over the place from CI/CD pipelines to Kubernetes pods. And they all suffer from the same problems. It's easily my least favourite thing about the DevOps modernisation of what would have been random sysadmin shell scripts 10 years ago. Frankly I'm not convinced these YAML files are any more readable nor less brittle than the duct tape we wrote in #!/bin/sh before.
> CDK is on the right track, but even there it's a mess, again for the sake of hiding complexity: constructs and deployment. Where did One thing well and Keep it simple stupid go? :)
As I'd written elsewhere, I think CDK is aimed at a subtly different audience. CF, TF, etc are more focused around the sysadmin side of DevOps, whereas CDK are more focused around the application developers end of the DevOps tool chain. That's not to say sysadmin guys can't use CDK, but rather than CDK doesn't just aim to deploy infra, it embeds and deploys the serverless applications (like lambda code) as well. It's more akin to the "full stack" style of developers and not every team nor individual who's job it is to manage cloud infra is going to be an application developer. Particularly in larger organisations. So there definitely is a place for cloud infra stacks to be described in less sophisticated languages (even if it doesn't appeal to you and I personally).
> For one, it's closer to a proper programming language as opposed to straight up data interchange format. Sure if you write it in YAML than you can take advantage of variables but YAML's syntax for variables is pretty gross.
I think that's what AWS CDK[0] and Terraform's CDKTF[1] are trying to solve.
Given the context of your example, I'd liken Terraform to CSS and CloudFormation to HTML; CDK/TF to Javascript. It's not a great analogy, but Terraform as is right now is just close enough to a programming language to deceive you into treating it like one. But it really isn't and those issues become glaringly clear the more you use it.
Newer versions of Terraform are much better. I think they went v1.0 at the right time. But I do agree that there are still plenty of warts in TF compared to a "proper" programming language. However TF 1.0 is still easily far more composable than CSS currently is. If anything, YAML (and thus CloudFromation) is more equivalent to CSS than TF is given YAML's support in the spec for variables, templates, etc.
I'm yet to try Hashicorps CDKTF but from what I've used of CDK it felt like the audience was a little different to those that would use TF. CDK feels more for orgs that would have the same team who write the application code (eg lambdas) also write the infra, a bit like Serverless (sls) is. Whereas Terraform tends to be more suited for orgs that like different teams managing infra from those managing application development. Generally speaking of course.
Ultimately all the above tools work fine for production systems so its often just a question of preference.
Out of interest, do you find yourself writing actual software with CDK stacks integrated, or is it more accurate to say the CDK is just a stand-alone bit of code purely for deploying infrastructure?
I'm definitely in the latter camp, which is something I find frustrating. I get that for a developer the syntax familiarity might make CDK easier, but for me as a non-developer the pain of groping around the terrible documentation and learning how classes are supposed to be used far outweighs the annoyance of fixing YAML indentation.
Ultimately I worry people are jumping on the "true IaC" bandwagon without acknowledging that if their infrastructure is supposed to be somewhat static and immutable, a declarative language might actually be better.
I strongly suspect that these CDKs are not very well designed. In particular, what I want is something that lets me generate YAML/etc in a type-safe fashion. That YAML is then the input for an engine which reconciles the desired state with the actual state (a la Terraform or
cloudFormation). The idea is that the “real programming language” layer just allows us to DRY our YAML. For a use case like this, we don’t need inheritance or methods, but just structs, maps, arrays, and functions; however, these CDKs typically index pretty hard on inheritance and generally make things more complicated than necessary.
It's important to note that Terraform doesn't compile down to CloudFormation stacks like CDK does. Terraform providers instead call AWS APIs directly. This should allow Terraform much more composability than CDK for the very points you've listed unfortunately Terraform does still compile down to a fucking JSON state -- which is easily my biggest complaint about Terraform.
My issue with anything that ultimately compiles to a JSON or YAML state/config is that you lose the dependency tree (or your dependency tree becomes rigidly defined by the way the transpiler converts your code into JSON). It causes so many problems on any large project that ultimately the only solution is to break the project up into smaller distinct projects within the same git repository. Which is basically the same solution to working with JSON/YAML CF stacks directly.
If someone can create a language (or SDK for $PROGRAMMING_LANG) that then worked with AWS APIs directly, (like Terraform), didn't just transpile back to JSON, and isn't as verbose as using boto3 directly -- well I think that product might stand a real chance of displacing Terraform.
I've got a lot of strong thoughts on how this could be done right so I did consider having an attempt at it myself using the shell scripting language I'd designed as a rough foundation. But having a full time job already, two young kids, and a growing in popularity open source project (namely my aforementioned $SHELL), I realised any Terraform competitor I did create would be doomed to either never being maintained, or the project would literally burn me out.
Neither system deals well with large projects, but my CF criticisms weren’t predicated on the project size. So at the end of the day, CF has all of the problems TF has and then some.
But I do think that it would be really interesting to build an IaC project, and you’re right that it probably would be doomed. :)
That is certainly my impression with CDK. An even bigger heartache is the fact that I am writing a Python "program", but then have to use a JS binary to execute the deployment. Having the CDK synth/deploy functionality exposed via actual execution of the Python script (or even a built-in capability of the regular AWS CLI) would make much more sense to me.
Your suspicions are incorrect - I'd suggest trying the AWS CDK as it solves exactly the problem you want it to solve in the sense of providing strong type safety.
For one of my projects I use CDK (JS) with a JavaScript project and the CDK part does call limited parts of the project to lookup how many DynamDB tables to create and which kinds of which indexes to generate.
Also the API gateway configuration is generated based on registered endpoints within a larger application, but that's not being done directly in the CDK, but as a separate explicit step in the Makefile as a dependency of the CDK targets.
Disclaimer: I work for AWS, but not on anything related to this, other than using it.
