The former is what I had in mind.

So nothing in the spec prevents that, besides social convention/implementations not taking that into account?

ActivityPub question: why is it forbidden to change audience on an Activity after creation?