Remove spec.yml (#4456)

This PR removes the file .github/workflows/spec.yml because it is not used any more.

Author: eernstg