Announcing activitypub-testing: a test runner for ActivityPub Actors
A couple months ago, we started working on a conformance testing tool for ActivityPub servers, and now we're sharing an early version publicly for the first time.
I (bengo) blogged about this project for the first time about a week ago, and it implements some of the ideas I described in my more recent blogposts,Testing Ecosystem Wishlist and Use Test Specifications to Improve Testability of ActivityPub Requirements.
In this early alpha release, there are test cases for each of the eleven 'easy to test' requirements I categorized in Which ActivityPub Server Requirements are Easy to Test?.
- start trying out the
activitypub-testingagainst real-world activitypub actors and analyzing the results, whether via cli or via our browser-based tool
- be gentle if something is surprising to you.
- This is an early release and is full of weakly held opinions and agile design decisions. We expect to iterate on this a lot based on learnings from your feedback, adding more tests, implementing test specifications, etc. The repo has an issue tracker you can add issues to.
Example: Testing an Actor at onepage.pub
bengo created an issue activitypub-testing should interoperate with onepage.pub
then we did a sample Run 0 with an invocation like
npx activitypub-testing test actor https://onepage.pub/person/qfU45YyW4aDIG-lN_lHSS
one test gave a result with outcome
failed, which upon closer look didn't seem like a fair determination, so we improved the test to not be so picky
we did a Run 1 with the improved version of
activitypub-testingand took advantage of a new feature to provide proof of Authorization (required by onepage.pub when handling outbox POST requests)
activitypub-testing test actor https://onepage.pub/person/qfU45YyW4aDIG-lN_lHSS --input.authorization="Bearer `cat ~/.activitypub-testing/bengo_onepagepub_token`"
Analysis of Run 1 shows onepage.pub gets 8
passedAssertions which correspond to satisfying 8 conformance requirements.
Request for Comment (RFC)
What will be especially interesting is discussing the results of specific runs against specific actors, e.g. using
npx activitypub-testing test actor <your-actor-url>
Comment using ActivityPub
File an Issue
If you'd rather record a comment but without using ActivityPub, the next-best place to publish an issue is into the git repository's issue tracker, e.g. https://codeberg.org/socialweb.coop/activitypub-testing/src/branch/main/issues.