tests: add unit tests for bitmask functions