Skip to content

Apply iteration name provides to simple features as well#1238

Merged
leonard84 merged 3 commits intospockframework:masterfrom
leonard84:fix-1236
Jan 18, 2021
Merged

Apply iteration name provides to simple features as well#1238
leonard84 merged 3 commits intospockframework:masterfrom
leonard84:fix-1236

Conversation

@leonard84
Copy link
Copy Markdown
Member

fixes #1236

@codecov
Copy link
Copy Markdown

codecov Bot commented Nov 26, 2020

Codecov Report

Merging #1238 (1666b09) into master (31adcf7) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1238      +/-   ##
============================================
+ Coverage     76.56%   76.57%   +0.01%     
- Complexity     3748     3753       +5     
============================================
  Files           405      405              
  Lines         11418    11424       +6     
  Branches       1389     1389              
============================================
+ Hits           8742     8748       +6     
  Misses         2196     2196              
  Partials        480      480              
Impacted Files Coverage Δ Complexity Δ
...java/org/spockframework/runtime/IterationNode.java 100.00% <100.00%> (ø) 10.00 <1.00> (ø)
...ockframework/runtime/ParameterizedFeatureNode.java 78.18% <100.00%> (ø) 14.00 <1.00> (ø)
...org/spockframework/runtime/PlatformSpecRunner.java 92.79% <100.00%> (+0.03%) 75.00 <0.00> (ø)
.../org/spockframework/runtime/SimpleFeatureNode.java 100.00% <100.00%> (ø) 10.00 <1.00> (ø)
...main/java/org/spockframework/runtime/SpecNode.java 100.00% <100.00%> (ø) 11.00 <1.00> (ø)
.../runtime/extension/builtin/TempDirInterceptor.java 95.55% <100.00%> (ø) 15.00 <0.00> (ø)
...work/runtime/extension/builtin/TitleExtension.java 100.00% <100.00%> (ø) 2.00 <1.00> (ø)
...rg/spockframework/runtime/model/IterationInfo.java 90.47% <100.00%> (+1.00%) 11.00 <3.00> (+2.00)
.../spockframework/runtime/model/SpecElementInfo.java 92.85% <100.00%> (+0.85%) 15.00 <3.00> (+3.00)
...ockframework/junit4/JUnitDescriptionGenerator.java 70.00% <100.00%> (ø) 5.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 31adcf7...1666b09. Read the comment docs.

Copy link
Copy Markdown
Member

@Vampire Vampire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this is good and especially consistent.
The old way was also kind of a hack and it is questionable how and if it should be supported and if, how to do it consistently.
I extended your test example a bit, added @Unroll to parameterized and added a parameterized2 with no annotation for 1.3 and @Rollup for 2.0.
So we have:
test => simple feature
parameterized => unrolled feature (without the extension)
parameterized2 => uprolled feature (without the extension)

With 1.3 I get this:
grafik

With 2.0 master I get this:
grafik

With this PR I get this:
grafik

So in 1.3

  • it works on simple features name
  • it does not work on already unrolled DD-iterations (DD = data-driven)
  • it "works" on uprolled DD-iterations, unrolling them and defining the name, not appending

In 2.0 master

  • it does not work on simple features name
  • it does not work on DD-features name
  • it works on already unrolled DD-iterations, appending
  • it "works" on uprolled DD-iterations, unrolling them and defining the name, not appending

In this PR

  • it does work on simple features name
  • it does not work on DD-features name
  • it works on already unrolled DD-iterations, appending
  • it "works" on uprolled DD-iterations, unrolling them and defining the name, not appending

The original issue essentially was about difference between Spock 1 and 2, but with this PR it is still differente between the two and it is also inconsistent a bit in itself, for example it now works for the feature name of a simple feature but not for the feature name of a DD-feature.


@Retention(RetentionPolicy.RUNTIME)
@Target([ElementType.TYPE])
@Inherited
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just superfluous here, isn't it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it was c&p from the example code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, maybe it could be removed?

@leonard84
Copy link
Copy Markdown
Member Author

Is it really inconsistent? The sample extension explicitly sets feature.reportIterations = true so that it negates the @Rollup. Just adding a constant suffix is probably not the real usecase so I can't really comment on that. Also there wouldn't have been a need to use @Unroll for 1.3 as the unrolling was already done with the @Suffix extension.

@Vampire
Copy link
Copy Markdown
Member

Vampire commented Nov 26, 2020

The unrolled / uprolled is not so much the inconsistency but more showing that this is maybe more a hack than a proper feature.
The inconsistency is, that with this PR some feature names are affected (simple) and some not (data-driven).

@leonard84
Copy link
Copy Markdown
Member Author

Well the simple feature also represents a single iteration, whereas the parametrized feature does not have its own iteration, as it is only a container for the individual iterations.

@Vampire
Copy link
Copy Markdown
Member

Vampire commented Nov 26, 2020

That's about technical details and then it depends on the exact terminology used.
In Spock terms this is correct, in JUnit platform terms it is not as the container is also reported as test. ;-)

But I'm talking less about the technical correctness but more about the perceived correctness as seen by the user.
@ldaley as you reported the original issue, what is your opinion?

@ldaley
Copy link
Copy Markdown
Member

ldaley commented Dec 2, 2020

My usage of this in Spock 1 was a “hack” insofar as it was abusing this mechanism in order to augment test names with global parameterisation context.

We have a system where we parameterise entire test suites via system properties when launching, and desire to have the parameters be apparent in the reported test names. It's a very important aspect of our Spock usage.

Without understanding all of the details, I can see that a mechanism more focussed on augmenting the “reported test name” would be better than relying on this indirect mechanism. I don't think it would be a problem if that mechanism is different between 1 and 2.

@leonard84
Copy link
Copy Markdown
Member Author

@ldaley take another look (UnrolledFeatureMethods), you can now properly set the displayName of a feature via an extension, which you also could do in Spock 1.x, but that affected iteration naming as well.

to separate name and displayName. The new displayName is now used to
set the reported name.
The name field should now always contain the raw value
although existing extensions might still modify it.

fixes spockframework#1236

Undo
@ldaley
Copy link
Copy Markdown
Member

ldaley commented Dec 29, 2020

@leonard84 LGTM. Looking forward to using it.

@leonard84 leonard84 added this to the 2.0 milestone Jan 4, 2021
@leonard84 leonard84 merged commit 510e1ed into spockframework:master Jan 18, 2021
@leonard84 leonard84 deleted the fix-1236 branch January 18, 2021 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Iteration name provider does not apply to non parameterized tests in Spock 2

4 participants