Do We Need Cucumber For UI Test Automation?
I work on a team of highly skilled solution architects. Combined we have close to 100 years of test automation experience. We have seen ONE good Cucumber implementation in our lives by Aslak Hellesoy. Hopefully, this alone makes you reconsider using Cucumber without following the BDD process. If not, let’s talk about why your automation might be at risk.
It all starts with a simple question…
What is Behavior Driven Development?
Behaviour-Driven Development (BDD) is a set of practices that aim to reduce some common wasteful activities in software development:
– Rework caused by misunderstood or vague requirements.
– Technical debt caused by reluctance to refactor code.
– Slow feedback cycles caused by silos and hand-overs
BDD aims to narrow the communication gaps between team members, foster better understanding of the customer and promote continuous communication with real world examples.
Now that we understand that, let me ask…
Did you see the word tool or tools used a single time?
No, we didn’t.
This implies that BDD is NOT a tool, it is a set of practices.
Aslak Hellesoy, the creator of Cucumber says,
If you think Cucumber is a testing tool, please read on, because you are wrong.
There is a process to follow that involves many roles on the software team.
This process is called BDD. It’s what came out of that clique I mentioned. BDD is not a tool you can downloadThe World’s most misunderstood collaboration tool, Cucumber.io
I’m fortunate in that I’m a Solutions Architect (SA) and I get to talk to dozens of new customers and hundreds of automation engineers every year.
Here’s where it gets bad:
The common problem that I encounter with my fellow SAs…
is that almost nobody uses BDD as a set of practices.
No practices are implemented to remove all the waste and technical debt.
Instead, tools such as Cucumber, Serenity, SpecFlow are used to write automated tests and then we claim that we are “doing BDD”.
It gets worse:
The problem is that using BDD automation tools adds an extra layer of complexity and dependency to the test automation code.
If we aren’t using the BDD process for the actual advantages then all we are doing is adding extra complexity for no benefits 🙁
These are the problems that I see when a BDD tool is used for automation without actually following the BDD practices:
Problem 1: BDD tools create more dependencies
Let’s take a look at a diagram that shows all the dependencies that are added when using a BDD automation framework (I didn’t include all the other dependencies such as test runners and so on as they’re not related to BDD).
When you add a BDD tool to your automation suite, you have the BDD framework such as Cucumber used by Feature Files. Feature Files use Step Definitions and Step Definitions use Page Objects.
Hold on to that thought for a second…
What if you don’t use a BDD automation tool?
By not using a BDD tool, we can remove two extra dependencies.
Dependencies in software development are almost always bad. We want to limit the number of dependencies because each one is a chance for something to go wrong.
Most of the software development design patterns focus on dependency management…
Think Single Responsibility Principle or Open-Closed Principle.
In software development, we strive towards having our modules doing less while limiting the number of dependencies.
Nikolay Advolodkin
So why are we adding extra BDD tool dependencies to our automation if we aren’t using the process?
If you want to learn more of the automation best practices as well CI/CD and framework development then check out The Complete Selenium WebDriver with Java Bootcamp.
BDD tools help to create more readable code:
Yes, this is true. I would agree that a test written in good Gherkin syntax is very readable.
However, is it that big of a difference when compared to a non-BDD test like this:
I don’t believe it’s that drastic of a difference.
Is it worth it to take the risk of extra dependency management for slightly more readable tests?
It gets worse:
The other problem that seems to happen with the majority of the BDD tests is that they don’t follow actual BDD best practices.
Problem 2: BDD is being done by the wrong team!
I recently read an interesting article that pointed out another problem of using BDD Tools for automation. As most organizations currently do BDD, it’s the automation engineers that are simply converting manual tests to BDD tests, right?
What that ultimately means is that the completely wrong team is writing BDD.
The developers who are designing the code aren’t writing the test specifications. Even though they are most familiar with the system.
The BAs aren’t writing the specifications either because they view BDD as an automation tool. Even though the BAs are the ones that are the closest to the client.
It gets worse:
The BDD specs are being written by the isolated team that rarely communicates with the client and doesn’t have an intimate knowledge of the system being tested.
How does that make any sense?
Problem 3: BDD tools struggle with parallelization
As far as I’ve seen with BDD automation frameworks such as Cucumber or SpecFlow, parallelization with them is a problem.
You can only parallelize at the Feature File level. This means that you can only run as many tests in parallel as you have feature files. This is a major problem if you are trying to scale your test automation.
Here’s the secret:
If you can’t run in parallel, you can’t scale. Hence, you can’t get fast enough feedback. As a result, your automation program will fail, I can guarantee that.
Problem 4: Poor Gherkin is coded
Let me start by saying that I’m NOT a BDD expert by any means. In my career, I have written about 50 Cucumber tests. All in an attempt to help other teams understand how a correct Cucumber implementation should look. And I don’t plan on ever writing a production Cucumber test since I find it an unnecessary burden for little gain.
However, I read the documentation as I work with 100s of automation engineers every single year to resolve the problems that they encounter with Cucumber. So my Gherkin may not be perfect, but if you doubt my ideas, check the resources for yourself 😉
Here are some examples that I have seen in real life:
Here’s another one
The purpose of Behavior Driven Development scenarios is to document end-user behavior. User’s don’t come to our page to send keys, click buttons and synchronize on elements. Users come to our pages to login or book a hotel or check a reservation.
The danger in exposing UI interactions in the BDD tests is that any time any UI flow changes, you must update all tests to reflect that change. For example, if today your login doesn’t have a captcha, but tomorrow it does.
Even more dangerous are the exposed sleeps. Locally, 10 sec may be enough. In a cloud grid, it will not. Now every test that uses that functionality needs to be updated to wait longer. If this was in a single step of a page object, we would just need to update a single place.
Problem 5: Unnecessary Overhead
Every technology brings overhead with it because you need to learn and answer some of the following questions:
- How does it work?
- What are the best practices?
- How do I use it?
- How do I install it?
- …
Here’s a real example of some of the rules that you will need to learn if you want to do BDD correctly (there are many more than this):
Write all steps in third-person point of view
Write steps as a subject-predicate action phrase
Given-When-Then steps must appear in order and cannot repeat
…
https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/
The most important question that we need to ask ourselves is
Why are we trying to answer all of these questions? What is the problem that we are actually trying to solve?
For slightly better readability… Because our boss told us so…
Those don’t seem like valid reasons to spend extra time mastering another tool.
If you want to learn more of the automation best practices as well CI/CD and framework development then check out The Complete Selenium WebDriver with Java Bootcamp.
