Confirm List of Queues will be available

[danielrehmann]

Hey there,

we are currently evaluating if we want to use terraform-compliance in our application. One of our use cases is to ensure all necessary Servicebus Queues will be available after performing the terraform apply.
Our queue resources are generated by following TF code:

resource "azurerm_servicebus_queue" "sb" {
  for_each            = toset(var.servicebus_queue_list)
  name                = each.value
  resource_group_name = ".."
  namespace_name      = "..."
  enable_partitioning = true
  dead_lettering_on_message_expiration = true
  default_message_ttl = "P14D"
}

Where we pass a list of queues and expect them to be created.

We would like to write a scenario for each essential queue e.g. app-error

  Scenario: App Error queue exists
    Given I have azurerm_servicebus_queue defined
    Then it must contain name
    Then its value must contain app-error

This fails because we have multiple queues that have different names:

 Scenario: App Error queue exists
        Given I have azurerm_servicebus_queue defined
        Then it must contain name
        Then its value must contain app-error
          AssertionError: name property in azurerm_servicebus_queue did not contain app-error, it is set to ['another-queue']

We tried to add the name to the WHEN clause but it resulted in skipping test:

Scenario: ServiceBus configuration
        Given I have azurerm_servicebus_queue defined
	SKIPPING: Can not find app-error defined in target terraform plan.
        When its name is "app-error" regex
        ...

Could you guide us how we can test if the different queues exist after the apply?

Kind Regards,
Daniel

[ajmahoney]

Hi,
Had a similar situation and I think that name would be 'sb' as it's the resource name rather than the name setting and index should be app-error.

Hope this helps
Andy

[eerkunt]

Hi,

Sorry for the late response on this. Typical hectic work-life before xmas break.

I think the approach that @ajmahoney wrote is suitable here, you either need to use address ( to address exactly terraform reference ) or use index instead of the name. This really depends on the Azure Terraform Provider.

Would it be possible to share the plan.out (or plan.out.json) on this issue for me to dive a bit more deeper please ?

I will also try to create the problem on my local meanwhile.

[eerkunt]

Hello again,

Reproduced the problem on my local. I think now I understand what you mean :)

Currently I am creating 3 servicebus queues via ;

variable "servicebus_queue_list" {
  default = [
      "something",
      "somethingelse",
      "somethingdifferent"
  ]
}

resource "azurerm_servicebus_queue" "sb" {
  for_each            = toset(var.servicebus_queue_list)
  name                = each.value
  resource_group_name = "testgroup"
  namespace_name      = "testnamespace"
  enable_partitioning = true
  dead_lettering_on_message_expiration = true
  default_message_ttl = "P14D"
}

Of course the resource definition could be a bit different. Let's assume we normally want to ensure enable_partitioning must be set to true unless the somethingdifferent servicebus queue. The example HCL above sets ALL of them to true, so the tests must fail.

Here is my test ;

Feature: Test Scenario 
    Scenario Outline: App Error queue exists
        Given I have azurerm_servicebus_queue defined
        When its index is <name>
        Then it must contain enable_partitioning
        And its value must be <partitioning>

    Examples:
        | name               | partitioning |
        | something          | true         |
        | somethingelse      | true         |
        | somethingdifferent | false        |

and according to this test, here is the failure message I got, as expected ;

-          Failure: enable_partitioning property in azurerm_servicebus_queue.sb["somethingdifferent"] resource does not match with ^false$ regex. It is set to True.

Hope this helps ? Please let me know if you have any question about anything that I couldn't explain properly.

[frauklonk]

Hi @eerkunt thanks for looking into this issue!

The problem we have is that if a queue is not there the test is aborted and the test run is not marked as failed.

To emulate this the test case would have to look like this:

Feature: Test Scenario 
    Scenario Outline: App Error queue exists
        Given I have azurerm_servicebus_queue defined
        When its index is <name>
        Then it must contain enable_partitioning
        And its value must be <partitioning>

    Examples:
        | name               | partitioning |
        | notexisting | true |
        | something          | true         |
        | somethingelse      | true         |
        | somethingdifferent | false        |

and the set up would stay the same:

variable "servicebus_queue_list" {
  default = [
      "something",
      "somethingelse",
      "somethingdifferent"
  ]
}

resource "azurerm_servicebus_queue" "sb" {
  for_each            = toset(var.servicebus_queue_list)
  name                = each.value
  resource_group_name = "testgroup"
  namespace_name      = "testnamespace"
  enable_partitioning = true
  dead_lettering_on_message_expiration = true
  default_message_ttl = "P14D"
}

We need the test case to fail in order to detect missing queues. Any advise is appreciated!

[eerkunt]

Thanks for the explanation! 🎉

Now I understood your problem, sorry for the confusion so far.

It looks like this requires a small change in then is value contain step. I will make the change within #196.

After this change, you can achieve what you need by ;

  Scenario Outline: App Error queue exists
    Given I have azurerm_servicebus_queue defined
    Then it must contain name
    And its value must contain <name>

    Examples:

      | name                |
      | notexisting         |
      | something           |
      | somethingelse       |
      | somethingdifferent  |

  Scenario Outline: App Error queue have right partitioning
    Given I have azurerm_servicebus_queue defined
    When its index is <name>
    Then it must contain enable_partitioning
    And its value must be <partitioning>

    Examples:

    | name                | partitioning |
    | notexisting         | true         |
    | something           | true         |
    | somethingelse       | true         |
    | somethingdifferent  | false        |

Test, where you both ensure the queue name is created.

[eerkunt]

This issue was conflicting with #174, since different use cases were tried to be used within the same test steps. Additionally, there are other internal restrictions where some steps must run per resource. Due to this problem, I introduced a new step which will help this kind of use cases, where someone might filter out some values from multiple-resources and flatten/merge them within one place.

According to this development, the example I gave above will be obsolete, here is the working example with 1.1.0 ;

  Scenario Outline: App Error queue exists
    Given I have azurerm_servicebus_queue defined
    Then it must contain name
    And I flatten all values found
    And its value must contain <name>

    Examples:

      | name                |
      | notexisting         |
      | something           |
      | somethingelse       |
      | somethingdifferent  |

  Scenario Outline: App Error queue have right partitioning
    Given I have azurerm_servicebus_queue defined
    When its index is <name>
    Then it must contain enable_partitioning
    And its value must be <partitioning>

    Examples:

    | name                | partitioning |
    | notexisting         | true         |
    | something           | true         |
    | somethingelse       | true         |
    | somethingdifferent  | false        |

The only different is on the first Scenario.

    And I flatten all values found

This step will allow anyone to flatten all values found per resources into once place where further steps (either filtering or matching) can be applied against.

[eerkunt]

1.1.0 (and even 1.1.1) is released. Can you please have a try ? 🎉