Skip to content

feat: implement RPC signature verification for conveyor#12

Merged
kuny0707 merged 7 commits into
masterfrom
feat/rpc-signature-verification
Jul 11, 2026
Merged

feat: implement RPC signature verification for conveyor#12
kuny0707 merged 7 commits into
masterfrom
feat/rpc-signature-verification

Conversation

@ety001

@ety001 ety001 commented Jul 10, 2026

Copy link
Copy Markdown
Member

Adds the missing verification half of the JSON-RPC auth protocol that conveyor (the Go rewrite) needs to authenticate __signed requests from JS clients.

Background

conveyor is a Steem JSON-RPC 2.0 service that must verify __signed requests signed client-side by @steemit/rpc-auth + @steemit/libcrypto. steemutil had the signing side and the right crypto primitives but 4 gaps blocked auth. This PR closes all 4.

Changes

U1 — rpc.VerifySignedRpc (core)

New exported verifier mirroring @steemit/koa-jsonrpc's verifier (src/auth.ts:65-98) byte-for-byte, including order and exact wording of all error messages. Introduces AccountFetcher, AccountPostingAuth, KeyAuth so the chain-access mechanism is injected by the caller (conveyor uses steemgosdk get_accounts), not hard-coded. conveyor wires it as:

rpc.Validate(req, func(msg, sigs, acct) error { return rpc.VerifySignedRpc(msg, sigs, acct, myFetcher) })

U2 — wif compact signature helpers

DsteemSigToBtcec / BtcecSigToDsteem validate a 65-byte compact signature has byte0 ∈ [31, 35) and return it unchanged. The dsteem/libcrypto layout (recovery+31) and the btcec/decred layout (27+recoveryCode+4 for compressed keys) are byte-for-byte identical, so no offset arithmetic is applied. (VerifySignedRpc runs signatures through this before recovery.)

U3 — fix rpc.hashMessage nonce placement ⚠️ behavior change

The nonce was written into the inner (first) hash, diverging from @steemit/rpc-auth, which puts it in the outer (second) hash. Without this fix, Go could never verify a JS-signed request. Confirmed by byte-level comparison against the JS implementation:

JS rpc-auth      : 007310ac792eba274e44d1d26dda66be822df54ce0297074e0bf8e90528eadda
Go (before, BUG) : d62d5555ca4680bf7fd4bc51140f3fb10bc1e89f9732925fc5d4b23b24d6cc7e
Go (after)       : 007310ac792eba274e44d1d26dda66be822df54ce0297074e0bf8e90528eadda  ✅

This changes Sign's output. The prior digest was wrong (unverifiable against JS), so no correct consumer depends on it.

U4 — protocol/api chain types

ExtendedAccount, FollowCountReturn, FollowReturn, OrderBook, OrderPrice, FeedHistory for deserializing condenser_api / database_api responses. KeyAuth handles the nested-array key_auths wire format ([["STMxxx", 1], ...]) via a custom unmarshaler; reputation is kept as json.RawMessage to tolerate both string and number encodings.

Cross-language verification ✅

A signature produced by @steemit/libcrypto's signRecoverably (real signing implementation) is recovered to the correct public key by the Go path:

JS sign → Go DsteemSigToBtcec → RecoverPublicKeyFromSignature → STM7jNh5...  MATCH

This vector is fixed in TestVerifySignedRpc_JSVector.

Two corrections vs. the original task spec

During implementation, source-level + byte-level analysis against the authoritative JS implementations surfaced two errors in the task spec, both confirmed with the maintainer before proceeding:

  1. U3 — spec said "code is correct, only fix the comment"; analysis proved the code is wrong (nonce placement), so the logic was fixed.
  2. U2 — spec prescribed a +4 offset conversion; analysis proved the layouts are already identical, and the +4 would have made RecoverCompact reject valid signatures (byte0 < 27). Implemented as a validating identity transform instead.

Tests

  • go build ./... && go vet ./... && go test ./... all green (incl. -race, -count=1)
  • New tests cover every acceptance criterion: JS-vector verification, byte-tamper rejection, multi-key/multisig/weight/not-found/account-length errors (exact strings), round-trip conversion, wire-format deserialization.

ety001 added 7 commits July 10, 2026 14:27
Add the missing verification half of the JSON-RPC auth protocol that
conveyor needs to authenticate `__signed` requests from JS clients.

U1 - rpc.VerifySignedRpc:
  New exported verifier mirroring @steemit/koa-jsonrpc's verifier
  (src/auth.ts:65-98) byte-for-byte, including the order and exact
  wording of all error messages. Introduces AccountFetcher,
  AccountPostingAuth, and KeyAuth types so the chain-access mechanism
  is injected by the caller rather than hard-coded.

U2 - wif compact signature helpers:
  DsteemSigToBtcec / BtcecSigToDsteem validate that a 65-byte compact
  signature has byte0 in [31, 35) and return it unchanged. The dsteem /
  libcrypto layout (recovery+31) and the btcec/decred layout
  (27+recoveryCode+4 for compressed keys) are byte-for-byte identical,
  so no offset arithmetic is applied.

U3 - fix rpc.hashMessage nonce placement:
  The nonce was written into the inner (first) hash, which diverged from
  @steemit/rpc-auth (it belongs in the outer/second hash). Verified by
  byte-level comparison against the JS implementation: Go digest now
  matches the JS digest exactly. Without this fix, signatures on
  JS-signed requests could never verify.

U4 - protocol/api chain types:
  Add ExtendedAccount, FollowCountReturn, FollowReturn, OrderBook,
  OrderPrice, and FeedHistory for deserializing condenser_api /
  database_api responses. KeyAuth handles the nested-array
  key_auths wire format via a custom unmarshaler; reputation is kept as
  json.RawMessage to tolerate both string and number encodings.

Includes a cross-language test vector: a signature produced by
@steemit/libcrypto's signRecoverably is recovered to the correct public
key by the Go path, proving the two stacks interoperate.
- protocol/api: gofmt the Authority struct field alignment.
- rpc: remove misleading json tags from KeyAuth/AccountPostingAuth.
  These types are in-memory values returned by an AccountFetcher and are
  never JSON-deserialized (wire deserialization lives in protocol/api
  with custom UnmarshalJSON). Document the distinction so integrators
  pick the right type.
- rpc: cover the byte0 (recovery byte) tamper path explicitly. The prior
  loop skipped it and falsely claimed it was tested elsewhere.
VerifySignedRpc previously declared its own rpc.KeyAuth and
rpc.AccountPostingAuth types, duplicating the protocol/api.KeyAuth /
api.Authority types that already handle the condenser_api wire format.
The duplicates used int64 weights while protocol/api uses uint16, so
conveyor could not feed a deserialized account straight to the verifier
without manual conversion glue.

Drop the duplicate types and have AccountFetcher return api.Authority
directly. The rpc package now imports protocol/api (no import cycle:
steemgosdk already imports both packages together). conveyor can wire
the verifier with no type conversion:

    fetcher := func(name string) (api.Authority, error) {
        return accounts[0].Posting, nil
    }
    rpc.Validate(req, func(m, s, a) error {
        return rpc.VerifySignedRpc(m, s, a, fetcher)
    })
Two comments referenced DsteemSigToBttec (double t) while the actual
exported function is DsteemSigToBtcec. No code change.
These two files were not gofmt-clean on master (struct field alignment
and a trailing blank line). Run gofmt -w to bring them in line so the
whole protocol/api package passes gofmt. No logic change.
DsteemSigToBtcec was named for a format conversion that does not exist:
the dsteem/libcrypto and btcec/decred compact layouts are byte-for-byte
identical for compressed keys, so the function only validates. Rename it
to ValidateCompactSignature to describe what it actually does.

Drop BtcecSigToDsteem entirely. It was the inverse of a non-conversion,
duplicated the validator byte-for-byte, and had no production callers
(only a round-trip test that existed solely to exercise it).

No behavior change for the remaining function; callers in rpc/auth.go and
the tests are updated to the new name.
KeyAuth and AccountAuthEntry had custom UnmarshalJSON (to flatten the
condenser_api [["STMxxx", 1], ...] wire shape) but no MarshalJSON, so
re-serializing would emit a flat object instead of the array-of-pairs
shape. Add symmetric MarshalJSON implementations and round-trip tests.

Add malformed-input tests covering: non-array, single/three-element
arrays, non-string key, non-numeric weight. These surfaced an edge case
where a JSON null produced a spurious error instead of a zero value;
both UnmarshalJSON methods now short-circuit null as standard json does.
@kuny0707
kuny0707 merged commit 6a3bc7c into master Jul 11, 2026
3 checks passed
@ety001
ety001 deleted the feat/rpc-signature-verification branch July 12, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants