This post describes a proposed algorithm to unlink a forfeited vtxo from its in-round output.

The Bark SDK currently only supports in-round transfers between addresses of the same wallet as part of vtxo refreshes. The process is modeled in the following diagram:

user -> server: Attests to input and declares desired destination pubkey
server -> user: Gives hash and proposed round
user -> server: Cosigns branches of vtxo tree for proposed round
server: Broadcasts transaction
server -> user: Gives finalized round
user -> server: Gives forfeit signatures spending old vtxo exit to ASP hashlock
server -> user: Gives secret and new vtxo
user: Can now exit new vtxo with secret, or spend vtxo

Overview of in-round privacy issues

When senders refresh their vtxos, they can’t establish privacy across rounds from on-chain observers or the ASP.

On-chain, a stubborn ASP refusing to give the secret for a round output can force a user to exit their old vtxo, and if the ASP refuses to honor the spend of the new vtxo, they can cause that one to exit as well. This non-cooperation creates an on-chain link between the 2 transactions through their shared hash.

Off-chain, the ASP always sees the linkage from the hashlock binding the forfeit transaction to the new round outputs.

Addressing the privacy issues

To formalize the problem, we would like to avoid the ASP learning the relationships between input vtxos and round outputs, and find a way to maintain that privacy even when an ASP claims forfeited funds.

The first step I see that violates our 2 goals of on-chain privacy and privacy against ASP round linking is the attestation and output declaration step.

This step involves a user initiating an in-round transfer/refresh by submitting an attestation for an input and declaring an output public key. Since both pieces of information are given in one message, it trivially leaks the input/output relationship. Thus, our first goal is to separate these steps into 2.

The first step will take care of vtxo attestation, and the second step will take care of output declaration.

Vtxo attestation

In the vtxo attestation step, the user will no longer submit an output public key. A slot is an authorization to receive one output of a specific denomination in the round. The user chooses a fresh random slot_secret and gives the ASP slot_pubkey = slot_secret * G for a requested denomination of their input vtxo, where G is the curve’s generator.

Multiple denominations can be requested by providing different slot_pubkey[i] values, with each public key acting as an authorization bound to its registered denomination. The vtxo owner must sign these slot public keys and their denominations as part of their attestation to create a stronger binding between the two that will be helpful later on.

In terms of validation for this step, the ASP checks that the input is valid, that it hasn’t already been registered, and that the sum of all token denominations equals the input’s value after the agreed fees.

At this step, the ASP also establishes a MuSig2 signing session for the forfeit claim transaction and commits to the partial signature it will use. This is the transaction that lets the ASP collect from the forfeit output, not the transaction that creates that output. We call the partial signature scalar claim_secret and its public commitment claim_point = claim_secret * G. This scalar is specific to the fixed claim signing session. It is not the ASP’s private key. The claim transaction, keys, and both parties’ nonces are fixed here, so the signing challenge is fixed too. The participant calculates claim_point from the ASP’s public nonces and public key contribution using that challenge, then checks the point the ASP provides. We’ll put this point to use in the output declaration step.

Output declaration

For the output declaration step, we want to prioritize 2 things:

1. The ability for an ASP to verify that an output can only be spent with knowledge of a forfeit claim secret

2. A way for us to hide the forfeit secret used in our output construction

The existing implementation already meets the first requirement since the output is locked by the same hashlock needed to spend the forfeit output.

The second condition was a bit harder to meet. The challenge, under the existing implementation, is to create a construction such that the condition requiring the preimage of a hash could be obfuscated.

To simplify the task of obfuscation, we’ll first drop the hashlock from the forfeit output and introduce a new secret to lock our round output. Instead of the hashlock, we can use the ASP’s partial signature for the forfeit claim transaction. Going forward, we’ll call this secret claim_secret[i], where i represents the ith vtxo submitted for the round.

With this, we can bind the reveal of secret information to the action of the ASP’s forfeit claim. The invariant upheld is that when an ASP claims a forfeit, the participant learns a secret allowing them to spend the round output.

Now how can we use this secret to lock our output?

Locking our output

Similar to the original model, the participant creates a new keypair to use in their round output. Let’s call the private key in this keypair offset and the public key offset_pubkey = offset * G.

Instead of making offset_pubkey the final output public key we attach to our round output, we define the output public key as output_pubkey = claim_point[i] + offset * G, where claim_point[i] is the partial signature commitment associated with the participant’s input.

Once the participant learns claim_secret[i], they can calculate the private key output_secret = claim_secret[i] + offset needed to spend that output.

The participant also chooses a fresh, separate keypair to sign their branches of the vtxo tree. They already know this private key, unlike output_secret, which they can’t calculate until they learn claim_secret[i]. The tree-signing pubkey is submitted with the anonymous output request, and the participant checks that the proposed tree uses it for their output.

Hiding the claim point used

Hiding which claim point is used is not a straightforward task, so I leveraged one of the few cryptographic primitives known to me: ring signatures.

To hide which claim_point[i] is used when using slot_pubkey[i], the participant constructs a ring containing every branch pubkey registered for the requested denomination. For each candidate slot j in that denomination ring, we include 2 public keys slot_pubkey[j] and output_pubkey[i] - claim_point[j]. This turns the ring signature into a multi-key ring proof, reminiscent of Monero’s MLSAG (Multi-layered Linkable Spontaneous Anonymous Group).

Suppose Alice, Bob, and Carol each register a slot for the same denomination. Alice’s output proof includes all 3 slots, but only Alice knows the slot_secret and offset for the slot she uses. The ASP can verify the proof without learning which slot that was.

Implicit in the requirements for this mechanism is the ASP publishing each denomination pubkey collected to all participants. Each denomination pubkey will also have been signed by its vtxo owner in the vtxo attestation step, so it can’t be replaced by an ASP-derived value in this step.

In the ring, for one hidden slot i, the combined proof establishes knowledge of slot_secret and offset (since offset_pubkey = output_pubkey[i] - claim_point[i]). Slots from the same input share a claim_point, so the same offset works for those slots, but we still need the slot_secret for whichever slot we use. This approach is similar to the one used in the confidential transaction scheme described in Confidential Transactions For Dummies, Part 2 - Borromean Range Proofs.

The message we’ll sign in the ring signature is:

  "Bark private output declaration v1" ||
  round_id ||
  denomination

"Bark private output declaration v1" acts as a label for domain separation, the round ID marks it for that round and attempt, and the denomination makes it particular to that ring. Here round_id includes both Bark’s round_seq and attempt_seq, since a round can have more than one attempt.

We also include a nullifier, akin to a ‘key image’, in this construction to ensure the same real branch isn’t used twice. The ASP rejects an output registration if its nullifier has already been used for an accepted output in that round.

The combined proof thus contains:

slot_pubkey[i] = slot_secret * G
nullifier = slot_secret * nullifier_base
output_pubkey - claim_point[i] = offset * G

nullifier_base is a public curve point calculated with hash-to-curve from a domain-specific label, the round identifier, and the denomination ("Bark slot nullifier v1" || round_id || denomination).

nullifier_base can be shared by every candidate in a denomination ring. The proof contains a single nullifier = slot_secret * nullifier_base, which is used in every branch of the ring signature. Only in the real branch does the signer know the slot_secret relating both the slot public key and the nullifier to their respective bases. The other branches are simulated.

The challenge hashes the message, the ring’s full ordered list of public keys, the nullifier, and the branch’s computed nonce points for the slot, nullifier, and offset checks. The public keys are separate inputs to the challenge hash, not part of the message. Since they include output_pubkey - claim_point[j], this also binds the proof to the requested output public key.

The participant submits output_pubkey[i] (by itself), the tree-signing pubkey, the denomination, and the combined proof.

This request should also come back anonymously, so it must avoid reusing connections or identifiers that link it to input registration.

Once all participants have their exit information and the new round’s funding transaction has the required confirmations, the participant forfeits their old vtxo to the ASP in exchange for the claim_secret that will allow them to calculate output_secret. These confirmations are for the anchor funding the new round outputs, not the already-confirmed anchor behind the old input vtxo.

The ASP knows which input was forfeited and which claim_secret it released. The earlier anonymous output declaration does not tell it which output used that secret.

If the ASP withholds claim_secret[i], the participant will be forced to exit if they don’t want to lose money. If the ASP still wants to move forward, it will claim the forfeited exit, allowing the participant to extract claim_secret[i] from the completed claim signature and calculate output_secret.

This works because the combined MuSig2 signature scalar is the sum of the participant’s and ASP’s respective partial signature scalars, along with the tweak-derived value made from the script tree. Since the participant knows their own partial signature, they can calculate:

claim_secret = completed_signature_scalar - user_partial_signature - challenge*derive_sign(tweaked_agg_pubkey_parity_bit)*tweak

The participant also checks that the extracted claim_secret gives the registered claim_point.

If the exited vtxo is never claimed with a forfeit exit, the user is then able to claim their exit as described in the original model.

Privacy after a claim

The ASP already knows every claim_secret, but it doesn’t know which offset the participant chose. Spending an output reveals a signature, not output_secret.

The participant never reveals their slot_secret or offset, including during an exit. The combined proof must hide which slot they used.

Drawbacks

As with coinjoins, outputs can be linked to vtxo inputs when there is insufficient liquidity. If there’s one large-value vtxo and many small ones, heuristics may be able to link outputs.

The ASP can also impact privacy by providing fake liquidity with its own funds, reducing the anonymity set and making it easier to link real participant inputs and outputs.

The privacy afforded by this scheme also depends on other participants requesting the same denominations. A denomination with only one remaining input provides no input/output privacy. The choice of denominations therefore matters, even if every proof works as intended.

On the topic of denominations, I’ll also mention the problem of bundle fingerprinting. Although I ask that each participant submit their outputs over a fresh connection, timing analysis might bundle outputs submitted around the same time. This is bad because the sum of this subset may uniquely identify an input vtxo. Batching, randomized declaration timing, and avoiding uniquely identifying denomination decompositions can help with this, but the details of doing these things are out of scope for this post.

Another drawback is that the ring proof grows with the number of candidate slots, and splitting an input into smaller denominations creates more outputs and proofs. If those outputs later need to be exited, having more outputs can also mean more on-chain fees.

The ASP can also use its control over secret release to try to learn the relationship through timing analysis. For example, it could release one input’s claim_secret and watch for an output that immediately becomes active or is spent.

Notation

The notation I’ll use for single value commitment functions will be a little different from my other posts. In my previous posts I tried to abstract away as much information as I could and used commit(value) to represent commitments to ‘value’ (where ‘value’ is a scalar). Now because it will be relevant to the stuff I’ll show later, I’ll parametrize the commitment function to include the generator as its second argument. Thus we write commit(val, G) = val*G to denote a commitment to the value val with the generator G. I’d also like the reader to assume the commitment function is an elliptic curve multiplication between the scalar value being committed and a generator point on the curve.

The commitment scheme for a vector a is commit(a) = sum(commit(a[i], G_a[i]) for i..len(a)) where G_a is a public vector of different generators with length equal to a. We’ll use different generator vectors G_vec, where vec is the vector name, for each vector we commit to.

Stages of a zk argument

When talking about zk arguments that prove a relation between 2 hidden values, I think it’s helpful to break the process down into 5 stages. The statement, transformation, consistency check, operation, and verification stage. I’ll expand on these stages below.

Statement

The statement is the assertion being made about the “hidden” values. Hidden meaning we can only see commitments to the values we are making assertions about.

Transformation

The transformation hides the values in the statement such that the output is algebraically linked to the hidden values in a different way than the commitments are. The difference between transformed values and commitments is that transformed values can be used in operations that would be undefined on commitments. For example, multiplication is not defined between 2 commitments whereas it is defined between 2 transformed values.

Consistency Check

This step is where the verifier checks the transformed values are algebraically linked to the hidden values. This check is defined in a way that makes it nearly impossible to forge and is performed by applying a similar transformation on the statement’s commitment and checking that it equals the transformed value’s commitment.

Operation

The operation is where the transformed values are run through the operation in the assertion.

Verification

This step verifies the algebraic link between the operation’s result on the transformed values and the public result of the original values included in the statement.

The naive approach

In this section I’d like to describe what a naive inner product argument looks like when broken down into the stages listed above.

Statement

The statement for the inner product argument informally says “I would like to prove that 2 committed vectors, a and b, have an inner-product equal to a public scalar c.”

As part of this statement the prover needs to give the verifier:

• The commitment scheme
• A where A = commit(a, G_a)
• B where B = commit(b, G_b)
• c where c is the inner-product of a and b
• Any other pieces of information required for the proof that can’t be used to leak a and b
  • This will includes vectors R_a, R_b, and R_c

Transformation

We now have to find a transformation that hides vectors a and b. One requirement of this transformation is that it must be some function of an arbitrary value x supplied by the verifier. Another requirement is that it must be possible for the commitment of the transformation to be recreated using an expression of the commitment values provided in the statement.

The simplest transformation that allows for this adds a different random number to each value in the vector like so:

• a' = a*x + r1 where r1 is a vector of random numbers
• b' = b*x + r2 where r2 is a vector of random numbers

This transformation against a and b results in:

a' = [
  x*a[1] + r_a[1],
  x*a[2] + r_a[2]
]

b' = [
  x*b[1] + r_b[1],
  x*b[2] + r_b[2]
]

Consistency Check

For the verifier to acknowledge that these modified values are indeed derived from a and b, we must commit the result of our transformations, a' and b', and reconstruct each vector commitment with the commitments of the original vectors a and b. Let’s see how this process works for a.

Our original commitment A is defined as:

A = commit(a) = a[1]*G_1[1] + a[2]*G_1[2]

We must now use this commitment to re-create the commitment of a' which is defined as:

A' = commit(a') = (x*a[1] + r_a[1])*G_1[1] + (x*a[2] + r_a[2])*G_1[2]

With some algebraic manipulation, we can see that the terms that define A, a[1]*G_1[1] + a[2]*G_1[2], already exist within A':

A' =
(x*a[1] + r_a[1])*G_1[1] + (x*a[2] + r_a[2])*G_1[2] =
x*a[1]*G_1[1] + r_a[1]*G_1[1] + x*a[2]*G_1[2] + r_a[2]*G_1[2] =
x*a[1]*G_1[1] + x*a[2]*G_1[2] + r_a[1]*G_1[1] + r_a[2]*G_1[2] =
x*(a[1]*G_1[1] + a[2]*G_1[2]) + r_a[1]*G_1[1] + r_a[2]*G_1[2] =
x*(A) + R_a

where R_a = (r_a[1]*G_1[1] + r_a[2]*G_1[2])

We can see this transformation allows us to check the relation between the original commitments by multiplying our old commitment A by x and adding the commitment R_a onto it. The same process can be performed to verify b'.

Operation

Let’s look at what happens when we perform the inner product operation on transformed values a' and b', and see if the result is something we can construct with the commitment for the public product c:

a' = [
  x*a[1] + r_a[1],
  x*a[2] + r_a[2]
]

b' = [
  x*b[1] + r_b[1],
  x*b[2] + r_b[2]
]

c' = inner_product(a', b') =  (
  (x*a[1] + r_a[1]) * (x*b[1] + r_b[1]) + # product of first row
  (x*a[2] + r_a[2]) * (x*b[2] + r_b[2]) # product of second row
)

=  (
  x^2(a[1]*b[1]) + x*(a[1]*r_b[1]) + x(r_a[1]*b[1]) + (r_a[1] * r_b[1]) + # Polynomial expansion of first row product
  x^2(a[2]*b[2]) + x*(a[2]*r_b[2]) + x(r_a[2]*b[2]) + (r_a[2] * r_b[2]) # Polynomial expansion of second row product
)

 =  (
  x^2*(a[1]*b[1] + a[2]*b[2]) +
  x*(a[1]*r_b[1] + a[2]*r_b[2] + r_a[1]*b[1] + r_a[2]*b[2]) +
  (r_a[1] * r_b[1] + r_a[2] * r_b[2])
)

This result can also be interpreted as a sum of dot products between the various components of a' and b':

c' = inner_product(a', b') = (
  x^2*(a dot b)
  + x*(a dot r_b + r_a dot b)
  + (r_a dot r_b)
)

Verification

Our goal for this stage is to identify how we can re-create the commitment of c' using the commitment of our public product c. Algebraically we can see the terms needed to create the original inner product a[1]*b[1] + a[2]*b[2] in c as the coefficient for x^2.

The other terms (a[1]*r_b[1] + a[2]*r_b[2] + r_a[1]*b[1] + r_a[2]*b[2]) and (r_a[1] * r_b[1] + r_a[2] * r_b[2]), with coefficients x^1 and x^0 respectively, are also attached to this value. This means we’ll need commitments for those as well.

Since none of these coefficients rely on the verifier-supplied x, we can calculate them before x is ever given and are what we define to be R_c[1] and R_c[2] in the statement.

With these additional terms R_c[1] and R_c[2], we can verify the inner product by checking the following equality:

• commit(c', G_c) = x^2*(commit(c, G_c)) + x*(R_c[1]) + R_c[2]

Optimizations

The rest of this post will detail optimizations for the inner-product argument presented by the paper ‘Efficient Zero-Knowledge Arguments for Arithmetic Circuits in the Discrete Log Setting’ (aka ‘BCC+16’). I took the time to break down the structure of this zk argument into stages because I wanted to frame each optimization as targeting a specific stage. Once the core optimization is understood, the rest of the actions taken can be thought of as compensating for the core change in the other stages.

Optimization 1 - Transformation step

The first step of the scheme we wish to optimize targets the transformation step. To rehash, the transformation step takes in vectors a and b and spits out equal size transformed vectors a' and b'. Since researchers in this field generally don’t want proving steps to involve returning data proportional in size to the input, a scheme was devised to reduce a' and b' into a singular scalar each. As we’ll see later on, using this optimization alone won’t allow us to make a correct inner product argument, but when mixed with optimization 2, it works.

Focusing back on this optimization, we intuitively know that this mysterious new transformation needs to involve all the terms of the input vector. Otherwise, we wouldn’t be able to verify the inner-product since it’s a function of every element. So now we ask, “What’s the simplest way to consolidate the elements of a vector into a single value?”

Hopefully your mind went to the same operation mine did. It’s one of the simplest operations math has to offer – addition. Now let’s see the impact of making the transformation the sum of all elements in a vector.

One thing we get for free when we do this is the elimination of the per-element random term. If the first 2 elements are summed, we can also think of the random elements associated with each term as being summed. Thus, all random elements can be consolidated into one element.

Also, because it will make things easier later on, let’s add the random value for each vector as the 1st element of the vector. Since our transformation will sum it up whether it’s attached to one element or if it’s another element itself, we take the route that allows us to write the simplest notation.

Here’s what the output of this new transformation looks like:

a' = [
  a[1] + x*a[2] + x*a[3]
]

b' = [
  b[1] + x*b[2] + x*b[3]
]

If the above looks weird to you, you might be onto something. Simply transforming the terms like this with a simple addition is not secure.

If the verifier only checks the aggregate, then different underlying vectors that lead to the same aggregate can be trivially forged. For example, if the check only depends on a[2] + a[3], then replacing a[2] with a[2] + 1 and a[3] with a[3] - 1 gives the same transformed value. If values were allowed to be forged this way, the verifier would no longer have probabilistic confidence that the original vector was committed, since it could now be a different vector that was trivially assembled to have the same sum.

Fortunately we have a fix for this. We just make sure every row has a different power of x applied to it before they are all summed together:

a' = [
  a[1] + x*a[2] + x^2*a[3]
]

b' = [
  b[1] + x*b[2] + x^2*b[3]
]

Consistency check

Now that we’ve transformed our hidden input, it’s time to check the output of the transformation for algebraic consistency. To do this, we must commit a'. But how do we apply G_a[1] to the a1 part, G_a[2] to the a2 part, and G_a[3] to the a3 part if all three values have been merged into a' and can no longer be separated?

With some thought, someone eventually came to the realization of trying to mirror the operation performed on the elements onto the generators that correspond to those elements. Let’s see what happens when we use G_a' = G_a[1] + G_a[2] + G_a[3] as our new generator.

I’m going to leave the new generator for a', G_a', broken down into a sum with the individual generator terms from the statement to allow us to see what happens to all the individual terms in the result a'*G_a' algebraically:

a' = [
  a[1] + x*a[2] + x^2*a[3]
]

commit(a') = (a[1] + x*a[2] + x^2*a[3]) * (G_1[1] + G_1[2] + G_1[3])
= a[1]*G_1[1] + a[1]*G_1[2] + a[1]*G_1[3] + x*a[2]*G_1[1] + x*a[2]*G_1[2] + x*a[2]*G_1[3] + x^2*a[3]*G_1[1] + x^2*a[3]*G_1[2] + x^2*a[3]*G_1[3]

Remember our goal to prove the expression for commit(a') contains the commitment to the original vector commit(a) = a[1]*G_a[1] + a[2]*G_a[2] + a[3]*G_a[3]. To work towards this goal, let’s try to at least group the relevant terms together and keep it at the front of the expression:

commit(a') = a[1]*G_a[1] + a[1]*G_a[2] + a[1]*G_a[3] + x*a[2]*G_a[1] + x*a[2]*G_a[2] + x*a[2]*G_a[3] + x^2*a[3]*G_a[1] + x^2*a[3]*G_a[2] + x^2*a[3]*G_a[3]
= (a[1]*G_a[1] + x*a[2]*G_a[2] + x^2*a[3]*G_a[3]) + a[1]*G_a[2] + a[1]*G_a[3] + x*a[2]*G_a[1]  + x*a[2]*G_a[3] + x^2*a[3]*G_a[1] + x^2*a[3]*G_a[2]

The problem with this expression is that even though the first grouping, (a[1]*G_1[1] + x*a[2]*G_1[2] + x^2*a[3]*G_1[3]), has the same terms of our commitment, it also adds a different power of x to each term. This is a problem because we don’t have individual elements needed to re-create that expression. The whole point of this new transformation is so we could eliminate the need for per-element commitments! This leaves us at an impasse.

The brilliance of the paper ‘Efficient Zero-Knowledge Arguments for Arithmetic Circuits in the Discrete Log Setting’ is that it found a way to tackle this problem. Their insight is that even though we don’t know the individual commitments of the original a vector, there might be modifications they could make to the individual elements in the G vector to force the expression into a checkable form.

Their answer to this was to have the verifier pre-multiply each G_i[i] with the inverse of the x that would be attached to the corresponding element in the first grouping to make the xs cancel out. Concretely, they would modify G_1 = [G_1[1] + G_1[2] + G_1[3]] to G_1' = G_1[1] + (x^-1)*G_1[2] + (x^-2)*G_1[3].

With G_1', the first grouping would turn into the original commitment and allow the verifier to re-construct commit(a') using commit(a). The other terms impacted are cross-terms anyway so we can just give the commitments of the x co-efficients at the beginning and have the verifier adapt it with the x they choose during the transformation step for verification.

Let’s see how this works in practice:

a' = [
  a[1] + x*a[2] + x^2*a[3]
]
commit(a') = (a[1] + x*a[2] + x^2*a[3]) * (G_1'[1] + G_1'[2] + G_1'[3])

= (a[1] + x*a[2] + x^2*a[3]) * (G_1[1]*(x^0) + G_1[2]*(x^-1) + G_1[3]*(x^-2))

= a[1]*G_1[1] + a[1]*G_1[2]*(x^-1) + a[1]*G_1[3]*(x^-2) + x*a[2]*G_1[1] + x*a[2]*G_1[2]*(x^-1) + x*a[2]*G_1[3]*(x^-2) + x^2*a[3]*G_1[1] + x^2*a[3]*G_1[2]*(x^-1) + x^2*a[3]*G_1[3]*(x^-2)

= (a[1]*G_1[1] +  x*a[2]*G_1[2]*(x^-1) + x^2*a[3]*G_1[3]*(x^-2)) + a[1]*G_1[2]*(x^-1) + a[1]*G_1[3]*(x^-2) + x*a[2]*G_1[1] + x*a[2]*G_1[3]*(x^-2) + x^2*a[3]*G_1[1] + x^2*a[3]*G_1[2]*(x^-1)

# Let's simplify the first grouping
= (a[1]*G_1[1] +  a[2]*G_1[2] + a[3]*G_1[3]) + (x^-1)*a[1]*G_1[2] + (x^-2)*a[1]*G_1[3] + x*a[2]*G_1[1] + x^-1*a[2]*G_1[3] + x^2*a[3]*G_1[1] + x*a[3]*G_1[2]

# Now let's group the cross terms by their `x` coefficient
= (a[1]*G_1[1] +  a[2]*G_1[2] + a[3]*G_1[3]) + (x^-2)*a[1]*G_1[3] + (x^-1)*(a[1]*G_1[2] + a[2]*G_1[3]) + x*(a[2]*G_1[1] + a[3]*G_1[2]) + x^2*a[3]*G_1[1]

As you can see from the calculation above, the first grouping is the commitment to the original vector a.

Operation

a' = [
  a[1] + x*a[2] + x^2*a[3]
]

b' = [
  b[1] + x*b[2] + x^2*b[3]
]

a'*b' = (a[1] + x*a[2] + x^2*a[3]) * (b[1] + x*b[2] + x^2*b[3])
= a[1]*b[1] + a[1]*x*b[2] + a[1]*x^2*b[3] + x*a[2]*b[1] + x*a[2]*x*b[2] + x*a[2]*x^2*b[3] + x^2*a[3]*b[1] + x^2*a[3]*x*b[2]  + x^2*a[3]*x^2*b[3]
= x^4*(a[3]*b[3]) + x^3*(a[2]*b[3] + a[3]*b[2]) + x^2(a[1]*b[3] + a[2]*b[2] + a[3]*b[1])+ x*(a[1]*b[2] + a[2]*b[1]) + a[1]*b[1]

One may think a similar verification operation to the consistency check where the verifier replaces G with G' can now be performed to eliminate the x coefficients that prevent the original dot product (a*b = a[1]*b[1] + a[2]*b[2] + a[3]*b[3]) from forming, but they’d be understandably incorrect. Using a new generator G_c' for the commitment of the public product is not sufficient. Let’s see why.

The issue

Now that we have = x^4*(a[3]*b[3]) + x^3*(a[2]*b[3] + a[3]*b[2]) + x^2(a[1]*b[3] + a[2]*b[2] + a[3]*b[1])+ x*(a[1]*b[2] + a[2]*b[1]) + a[1]*b[1], you’d think we could eliminate the coefficients of the x^2 on a[2]*b[2] and x^4 on a[3]*b[3], and in fact we can! All we have to do is define G_c' to be G_c' = G_c + x^-2*G_c + x^-4*G_c. Each generator term will generate a copy of all terms in c' scaled by the coefficient on that generator term. Each term in the resultant expression of c'*G_c' that isn’t a term of the dot product of a and b should therefore be treated as a cross-term.

The problem with this is that this will generate 2 additional terms without an x coefficient that we’d have to commit to. To better visualize this process, let’s work it out:

c' * G_c' =
  (x^4*(a[3]*b[3]) +
  x^3*(a[2]*b[3] + a[3]*b[2]) +
  x^2(a[1]*b[3] + a[2]*b[2] + a[3]*b[1]) +
  x*(a[1]*b[2] + a[2]*b[1])
  + a[1]*b[1]) * (
    G_c + x^-2*G_c + x^-4*G_c
  )

= ... honestly an expression that's too long for me to want to work out

Let's just see which terms would be left over without an `x` coefficient:

(a[1]*b[1])*G_c +  x^2(a[1]*b[3] + a[2]*b[2] + a[3]*b[1])*x^-2*G_c  + (x^4*(a[3]*b[3]))*x^-4*G_c =

a[1]*b[1]*G_c +  a[1]*b[3]*G_c + a[2]*b[2]*G_c + a[3]*b[1]*G_c + a[3]*b[3]*G_c =
(a[1]*b[1]*G_c + a[2]*b[2]*G_c + a[3]*b[3]*G_c) + a[1]*b[3]*G_c + a[3]*b[1]*G_c  =

Since we have additional terms a[1]*b[3]*G_c and a[3]*b[1]*G_c not scaled by x, this introduces a security concern. If the statement claims the dot product is 5, a prover could supply malicious vector values whose actual dot product is 3, while the additional unscaled cross-terms contribute the missing 2. The verifier would only see the combined value 5 and could not distinguish it from the claimed dot product.

Optimization 2 - Negative exponents on b

The next optimization is small but powerful. It allows us to fix the problem preventing us from running with the previous optimization. The insight for this optimization is using negative x exponents for the transformation step of b, and only b.

Instead of b' being defined as:

b' = [
  b[1] + x*b[2] + x^2*b[3]
]

We make it:

b' = [
  b[1] + (x^-1)*b[2] + (x^-2)*b[3]
]

To accommodate this change we also need to change how we modify the generator for the consistency check of b'. Instead of G_b' = G_b[1] + (x^-1)*G_b[2] + (x^-2)*G_b[3], we’ll define G_b' as G_b' = G_b[1] + x*G_b[2] + x^2*G_b[3]. Now, because the xs of a' and b' will cancel out, we’ll get the following as the dot product:

a' = [
  a[1] + x*a[2] + x^2*a[3]
]

b' = [
  b[1] + (x^-1)*b[2] + (x^-2)*b[3]
]

a'*b' = (a[1] + x*a[2] + x^2*a[3]) * (b[1] + (x^-1)*b[2] + (x^-2)*b[3])
= a[1]*b[1] + a[1]*(x^-1)*b[2] + a[1]*(x^-2)*b[3] + x*a[2]*b[1] + x*a[2]*(x^-1)*b[2] + x*a[2]*(x^-2)*b[3] + x^2*a[3]*b[1] + x^2*a[3]*(x^-1)*b[2]  + x^2*a[3]*(x^-2)*b[3]
# Now let's group the inner-product terms in the front of the expression
= (a[1]*b[1] + x*a[2]*(x^-1)*b[2] + x^2*a[3]*(x^-2)*b[3] ) + a[1]*(x^-1)*b[2] + a[1]*(x^-2)*b[3] + x*a[2]*b[1] + x*a[2]*(x^-2)*b[3] + x^2*a[3]*b[1] + x^2*a[3]*(x^-1)*b[2]
= (a[1]*b[1] + a[2]*b[2] + a[3]*b[3] ) + (x^-1)*(a[1]*b[2]) + (x^-2)*(a[1]*b[3]) + x*(a[2]*b[1]) + (x^-1)*(a[2]*b[3]) + x^2*(a[3]*b[1]) + x*a[3]*b[2]
= (a[1]*b[1] + a[2]*b[2] + a[3]*b[3]) + (x^-2)*(a[1]*b[3]) + (x^-1)*(a[1]*b[2] + a[2]*b[3]) + x*(a[2]*b[1] + a[3]*b[2]) + x^2*(a[3]*b[1])

In our verification step we just need to provide 4 cross terms (the coefficients on x^-2, x^-1, x, and x^2) and no modification of the committed dot product is needed!

Optimization 3 - Recursion

This optimization provides the biggest reduction in communication cost and it’s only possible with a slight modification of optimization 1. In this modified version, we don’t smush all the elements into 1 expression, we smush them into groups instead. So this:

a' = [
  a[1] + x*a[2] + x^2*a[3]
]

Becomes this:

a' = [
  a[1] + x*a[2],
  a[3] + x*0
]

Keeping that in mind, the main idea of this recursion-focused optimization is having the prover replace giving a' and b', and instead ask the verifier to create the commitments of A' = commit(a') and B' = commit(b') like they would in the consistency check. The prover then withholds the values of a' and b' and just gives the verifier c'. The prover also commits to c’ as C’. Using C, the cross-term commitments fixed before x, and the verifier’s challenge x, the verifier checks that C’ is the correct transformed inner-product commitment.

The assertion of whether the inner-product of a' and b' equals the prover-supplied c' is still unproven. The prover then asks the verifier to perform another inner-product argument using A', B', and c' to prove this assertion instead of having the verifier receive and perform the operation on a' and b' directly.

Why skip the operation step?

Because we group each cross-term by their x co-efficient, and the number of different cross-terms with different x coefficients scales with the size of a' and b', this step aims to reduce the number of cross-terms by reducing a' and b'.

In our 3 element example, since we only send the cross-terms needed to make A' instead of a' directly, we don’t have to worry about the increase in the number of elements of a' with the grouping scheme. The idea is we can defer the inner-product operation of a' and b' and prove it with the operation of smaller a'' and b'' vectors instead.

It may be easier to reason why this works using an inductive framework. If we can assert the product of preimages for A and B with knowledge of c = inner_product(a, b), we should also be able to assert the product of preimages for A' and B' with knowledge of c' = inner_product(a', b'). The only part we need to be careful about is properly linking A, B, and C to A’, B’, and C’.

Conclusion

I’m over explaining the inner-product argument from the cited paper here so I’m going to wrap it up here. In a future post I’ll probably try to make a part 2 where I explain the bulletproofs version of the inner-product argument.

In this post I’ll delve into what the vectorized product argument is and how it works. I was going back and forth trying to decide whether to group in the inner product argument here, but I now think the two concepts are different enough to warrant independent posts. Also, I suggest reading my last two posts on product arguments before reading this one since I assume knowledge of those explanations to simplify what’s written here. This argument is thus framed as an extension of sorts.

Some things to know

Maybe I should have brought this up in my previous post, but I am using 1-based indexing when using array notation on sets or vectors. I also tend to implicitly assume the role of prover sometimes so know if you’re reading a snippet written in first person, I’m probably discussing a prover action.

Given that we’re working with vectors now and matrices going forward, I’d also like to mention that I refer to the ith column of matrix A as a_i and the ith row as a[i]. I’ll always use array notation when indexing into those rows or columns so the first element of the first column will be denoted as a_1[1], and the first element of the first row will use a[1][1].

Finally, as usual, I avoid using Pedersen commitments in my writing in exchange for a simpler mental model but Pedersen commitments are what’s used in practice. I believe once you understand the core of my post, extending to Pedersen commitments isn’t too big of a leap which is why I omit it. I may use them in my explanations in the future, but I feel like they’ll impede understanding in these explanatory posts more than they’ll help.

Vectorized product argument (Hadamard argument)

The difference between proving a Hadamard product as opposed to a simple product between integers is that instead of proving a product of committed values, we’re now proving multiple different products corresponding to different sets of committed values.

To ease into our understanding of the vectorized product argument, let’s try to create a product argument using vectors, where all vectors only have one element. If in the regular product argument we’re proving product(a[i]) for i=1..n is equal to b, then for the vectorized approach we can make each preimage a[i] its own one-dimensional vector [a[i]]. Instead of proving b is the product of secret preimages a[1] and a[2] like in the original product argument, the prover will now have 2 secret preimage vectors [a[1]] and [a[2]]. The verifier is then given 2 vectorized commitments [A_1] and [A_2]. Everything else in the proving process for the product argument is the same.

How does this approach extend to vectors with multiple elements?

First, let’s add a restriction and say each vector has to be the same size. If the vectorized product argument can be seen as proving multiple product arguments at once, this restriction will force every product argument to have the same number of committed factors.

Our next step is to group terms of the same position across product arguments. So if a[1][1] and a[1][2] are the secret terms for one product argument, and a[2][1] and a[2][2] are the secret terms for another product argument, then the prover would respectively group the first and second terms into secret vectors like so:

v_first_terms = [a[1][1], a[2][1]]

v_second_terms = [a[1][2], a[2][2]]

product_terms = [b[1], b[2]]

If you think of them as column-vectors and assembled them into a matrix [v_first_terms, v_second_terms, product_terms], each row would have the terms for its own product argument. Later in this post, whenever I refer to the row, I’m referring to the individual product arguments.

For each product argument, a[1][1] * a[1][2] = b[1] and a[2][1] * a[2][2] = b[2], we run the same product argument described in the last post.

To recap, in this naive construction of a Hadamard argument, the prover’s public statement to the verifier now asserts that we know preimages for commitment vectors v_1 = [A[1][1], A[2][1]] and v_2 = [A[1][2], A[2][2]] that have a public Hadamard product of product_terms = [b[1], b[2]].

Why is the Hadamard product important

Why bother with introducing vectors and calling it a ‘Hadamard’ product argument if we can accomplish the same thing without the whole notion of a vector? The answer lies in the fact that there is a major speedup we can leverage by aggregating terms.

The optimization

Let’s review the single value argument to help us discover how we can optimize the vectorized product argument through aggregation.

The product argument shown in Grokking Product Arguments Part 2 goes as follows:

• Prover makes statement which includes commitments to preimages a and public intermediate products b
• Prover also gives commitments to cross-terms
• Verifier gives prover an integer x
• Prover aggregates all as into one term AND all bs into another term using verifier’s x
• Prover gives aggregate terms a' and b' to verifier
• Verifier computes the product of a' and b' which should contain a*b as a term
• Verifier re-creates the commitment of a'*b' using the commitment of the public product and cross term commitments

In our naive vectorized approach, the statement includes commitments to each preimage a, but what if there was a way to aggregate commitments for each column vector into a single element – reducing the cost of sending n commitments in a vector to just sending one commitment?

The answer to that ended up being “yes,” which leads to another follow-up: whether a product argument can be constructed on the vector level using these single value vector commitments. As you can imagine, the answer to this is also “yes.” We’ll now focus on how the product argument can be done on the vector-level with single value commitments.

The new commitment function for vectors

We define vector_commit(v) for a vector v to be the sum of all elements in v after being run through the single value commitment scheme, with each value commitment in the vector using a different generator. In other words, a vector commitment can be thought of as the sum of its committed elements, where each element gets its own generator.

The commitment scheme that works for our purposes looks like this (where the generator used depends on i): commit(a_vec) = sum(commit(a[i], i)) for i=1..len(a_vec)

So now we get to figure out how all the other math follows from this new commitment scheme. Like usual, let’s work through an example of vectorizing, and then proving, 2 product arguments.

Example

Since we’re proving 2 product arguments, each vector will have 2 elements. We’ll also define each product argument to have 2 factors. Since we always set the number of vectors equal to the terms, you might think we only have 2 vectors. In actuality, it’ll lead to 3 vectors since there’s a vector for each term and the term we add in for randomness (referenced in part 2).

Let’s take a look at what each party has to perform at each step and make notice of the changes that come with the vectorized approach.

Statement

First, assuming the role of the prover, let’s define our 3 secret column vectors a_1 = [a_1[1], a_1[2]], a_2 = [a_2[1], a_2[2]], and a_3 = [a_3[1], a_3[2]]. We also assert that the product of all elements at row index i across the a vectors equals the ith element of the public product vector, b_n. If all column vectors were appended into a matrix, we can conceptualize elements at the same index across column vectors as rows.

As part of our statement to the verifier, we’ll make the commitments for these column vectors A_1 = A_1[1] + A_1[2], A_2 = A_2[1] + A_2[2], and A_3 = A_3[1] + A_3[2].

Cross terms

Since we’re assuming everything else works like the original product argument, we also give cross-term commitments for each row as if they were being proven independently.

Verifier x

Now comes the verifier’s turn to choose x. When they do, instead of just scaling every term with a power of x, the prover now returns a vector sum where they scale every column vector with a power of x and get a resulting sum vector a' like so:

a' = a_1 + x*a_2 + x^2*a_3

This process is also done with the rolling product vectors b_i:

b' = b_1*x + b_2

To mirror the paper, I’ll put the random masking vector at the end of the b list. So in this example, b_2 is the random vector and b_1 is the intermediate rolling product vector. We have no b_3 because that would just be the public product in the statement.

Verifying the vector

The tweaked sum vector a' still needs to be checked, just like the tweaked single values did. To do this check, we can re-create the commitment of sum vector a', A', using the commitments of individual column vectors A_i. This looks like A' = A_1 + x*A_2 + x^2*A_3. A similar process is then done for the bs in b', B = x*B_1 + B_2.

Like that, we’re still able to make this commitment scheme work for verifying derived vectors a' and b'!

Verifier computes product

The process continues with the similarities in that we compute the Hadamard product between a' and b'. The big difference is the last step where we have to re-create the commitment of the resultant vector a'*b' with previously committed values. As a refresher, the committed values we use for this are the column vector commitments that were in the statement, and the cross-term commitments given at the beginning of the exchange.

Also, we will be using the zero argument optimization introduced in Grokking Product Arguments Pt 2 since it still improves this version of the argument (which is why the exponent on the xs for b_i counted down above).

To solidify this product operation in our head, let’s work out the first row of this product vector below in terms of column vector elements:

• a'[1]*b'[1] =
• (a_1[1] + x*a_2[1] + x^2*a_3[1]) * (x*b_1[1] + b_2[1]) =
• x*a_1[1]*b_1[1] + x^2*a_2[1]*b_1[1] + x^3*a_3[1]*b_1[1] + a_1[1]*b_2[1] + x*a_2[1]*b_2[1] + x^2*a_3[1]*b_2[1]
• x^3*a_3[1]*b_1[1] + x^2*(a_2[1]*b_1[1] + a_3[1]*b_2[1]) + x*(a_1[1]*b_1[1] + a_2[1]*b_2[1]) + a_1[1]*b_2[1]

To verify this product contains the public product in the old single value approach, we would just re-create the commitment of a'*b' using the commitments we were given for each row. This would work, but there IS a more efficient way to do this. Let’s transition into our last optimization.

The cross terms

If we look at the breakdown of each row i in a'*b', we’ll see they’re ALL of the form:

• x^3*a_3[i]*b_1[i] + x^2*(a_2[i]*b_1[i] + a_3[i]*b_2[i]) + x*(a_1[i]*b_1[i] + a_2[i]*b_2[i]) + a_1[i]*b_2[i]

Every row has terms with the same powers of x, x^0, x^1, x^2, and x^3. If we re-defined our protocol to skip doing a per-row reconstruction of a'*b', and instead summed every row, we can group cross terms with the same x factor which means the prover could also aggregate the grouped cross terms sent at the beginning of the argument. This would lessen the number of cross terms the prover has to send, therefore reducing the communication cost.

Attacking our aggregate scheme

One problem that we’ve run into before is the vector for a prover to cheat if they only know the aggregate and not the individual terms. In our example, there is a way to only know the sum of rows for a'*b' and not know the individual Hadamard products for each individual row. By forgoing individual row checks, we are skipping validation of the individual product arguments and a malicious prover could forge a proof.

For example, the prover could assert that the product of row a[1] is b[1] and the product of row a[2] is b[2] but actually make the product of a[1] be b[1] - 1 and the product of a[2] be b[2] + 1. Since we’re only checking the sum of the product arguments in the naive approach, this would validate even though we lied about what the individual products were!

To harden our scheme against this attack, we ask the verifier to choose a new value y. We’ll then incorporate y into our sum by doing (a'*b')[i] *y^i. Once we weight the rows with y, the fake aggregate becomes something like (b[1] - 1)*y + (b[2] + 1)*y^2, which no longer matches b[1]*y + b[2]*y^2 except for unlucky choices of y.

The big question we have to ask now is where in our protocol should the verifier give us this y value?

If we asked the verifier to give the new y at the same time they gave x, this would stop us from aggregating the cross-terms since these coefficients to the x terms in the row sum now include a y term.

The verifier must now give y before the prover gives the cross term commitments for the aggregated row check. That way the prover can incorporate y into the cross term commitments and still have the verifier use x to scale what the product terms in the final sum (a'*b')[i] * y^i should be.

Link to inner product argument

If you look closely at the sum we performed in the step to reduce the number of final cross terms, you’ll see that it was an addition of all the elements in a vector that already represented an element-wise multiplication. Meanwhile, an inner-product is also the sum of elements in a vector resulting from an element-wise product. To prove we know the product of committed vectors a_i, we have computed the y-weighted inner product of a' and b'. We’ll leverage this fact in our next post.

Conclusion

The main idea behind the Hadamard argument is that it lets us take a bunch of product arguments with the same structure and prove them together. Each row can be thought of as its own product argument, but instead of sending every commitment and cross-term for every row separately, we stack matching terms into vectors and use vector commitments to keep communication small.

From there, the argument starts to look very similar to the single-value product argument. We still use x to aggregate terms inside each product argument, but after vectorizing, we now have multiple row-level derived products to account for. If we just added those row results together directly, a prover could potentially fake the split across rows while keeping the final aggregate looking correct. This is where y comes in. By weighting each row-derived product with a different power of y, the verifier makes it much harder for fake row values to cancel out in the aggregate.

This is also where the connection to inner products starts to show up. Once we multiply matching entries, scale them by powers of y, and add everything together, we are effectively doing a weighted inner product. So while the Hadamard argument starts as a way to prove element-wise products, the final aggregation step naturally leads us toward inner-product arguments.

In the next post, I’ll dig more directly into that inner-product view and show how the same aggregation ideas can be used as their own standalone argument.

After starting to draft a post about vectorized product arguments, I realized one of the concepts I was introducing would better be explained in the scalar context first. This concept is another way to make a product argument, something I wrote about in my last post. Both the method in my last post, and the method in this post, are featured in the paper I’m using as reference, ‘Efficient Zero-Knowledge Argument for Correctness of a Shuffle’.

I decided to give this product argument special treatment because it’s used as part of the vectorized Hadamard product argument, but it’s never explained in the simpler case of scalars. Understanding this will prove crucial for understanding the vectorized argument without banging your head against the wall for hours like I did. Know that I do take some liberties with my construction in an effort to make everything easier to follow, but the final result should lead to a very similar construction to the argument presented for Hadamard products in the reference paper.

Some things to know

Just like in my last post, I abstract away Pedersen commitments to focus on the meat of the ideas. If you’re having trouble generalizing the scheme to work with Pedersen commitments, feel free to leave an issue on this blog’s Github repo.

Also, in this post I use the word tweak a lot, so I thought I’d define what I mean when I use it before we get into the nitty-gritty. To quote Michael Folkson, who answers what it means in the context of public keys, and who in turn quoted Bitcoin Optech, “Tweaking a public key means to alter it with a value (the tweak) such that it remains spendable with knowledge of the original private key and tweak.” I use it more to mean performing an operation on a value to get another related value. It may be an abuse of the term, but I use it in this way throughout my post.

Alternative Proof Approach to single product argument

The single product argument involves proving that the product of numbers a[i] for i=1..n equals b without revealing any of the a[i]s. I covered the mechanics for this in my post ‘Grokking Product Arguments’, but now, I’d like to show an alternative way we can prove the same thing.

If you recall, the general approach for the product argument presented in part 1 was:

• have prover give a commitment to “cross-terms”
• have the verifier give the prover x
• have the prover make derived values a[i]' using a[i]s and x
• have the verifier multiply tweaked terms to get derived product b'
• have verifier re-create derived product using commitments

In the alternative approach to this argument, the prover proposes a new tweaking mechanism. Instead of tweaking each term a[i] into a[i]' and returning each a[i]' separately, we can aggregate our tweaked terms a[i]' before sending them to the verifier. Doing this would lower the number of distinct elements we have to give the verifier and turns n prover-given elements into 1. This aggregation is done by summing the tweaked terms a[i]' together like so: a[1]' + a[2]' + .... Believe it or not, this aggregation also opens the door for another optimization – removing the r[i] terms.

Because summing the terms together proves an effective way to hide the individual values, you can think of a[i+1] serving as the new r[i] for a[i]. One thing to note is that despite the prover’s ability to effectively hide the individual values, we would still like to add a random element r to not leak the x-scaled sum. Adding the random term r also helps adhere to our definition of, what the reference paper describes as, a ‘special honest zero knowledge argument’. I won’t go into the definition for what that is here, but it provides some guidelines on how to structure our protocol to be a special kind of zero knowledge argument.

So is that it? Optimization complete? Well, no. Some readers may notice that just replacing the r[i]s with a single r and summing the tweaked terms together will break one of the checks performed by the verifier. The verifier now can’t verify whether the prover knows every individual term that was committed to, or just the sum.

Usually the verifier would check each tweaked term x*a[i] + r[i] by using the commitments A[i]=commit(a[i]) and R[i]=commit(r[i]) returned by the prover like so x*A[i] + R[i], but now, if the verifier is just given a', the prover can fake knowing the individual preimages a[i]. So how do we fix this?

One way is to have the verifier give each term a unique coefficient x[i]. Doing this prevents the raw sum of a[i]s from being forged since the prover would have to find a' such that a'*G = R + x[1]*A[1] + x[1]*A[2] + ... + x[n]*A[n], without knowing the individual a[i]s and only knowing the aggregate sum. To solve this we’d need to solve the elliptic curve discrete log problem for which all modern EC crypto-systems are based. So pretty unlikely to happen.

Well there’s one more challenge we’d like to overcome. Requiring the verifier to choose n xs increases the number of elements the verifier will have to send the prover. In the literature this metric is known as the communication cost. Our next challenge is to try to reduce this communication cost, and lucky for us.

But before I get into that, I’d like to modify the protocol a little bit to hopefully simplify the next explanation. Including the r in the derivation can be a bit tedious and generally serves as another piece of information we have to keep track of. What the reference paper does is prepend r into the list of preimages a[i], turning r to a[1]. This would make the existing protocol stay mostly the same, except now, we can implicitly re-define n to be the number of elements in a including the r element. Alright, now let’s get back to how we’re going to tackle reducing the communication cost of the xs.

Instead of having the verifier pick n xs where n is the number of as, the verifier can give the prover one verifier-chosen x and have the prover derive the other coefficients from it. More concretely, the verifier asks the prover to define x[i] as x[i]=x^(i - 1) for i=1..n. That way the verifier only needs to give one value to the prover, x. Problem solved!

Ok, one last thing. In our previous post we had the distinction of the simple case and the case of rolling products. This product argument has no such distinction given the 2 element case balloons the number of as to 3 because of the implicit r. Since every case turns into the rolling case of len(a[i]) > 2, we’ll be using the same concept of intermediate products b[i] introduced in the previous post. b[i] being defined as b[i]=a[1]*...*a[i].

Also similar to our previous post, we will need to tweak these intermediate products to keep them secret. To do this, we’ll tweak them the same way we tweaked our list of preimages a. We’ll include a random element for b[1] and create a sum of intermediate products scaled by different powers of x.

Now, let’s run our explanation of the product argument back with this new and improved protocol.

What is the product argument?

To reiterate from my first post, a product argument is a protocol between a prover and a verifier where the prover aims to convince a verifier that the preimages of n committed values, a[i] for i<=n, have a public product b. Just like in my previous post, I’ll be focusing on a simple case to better cement the idea with an example. This simple case will be when n=3 since we’ll need a random term to be included even when we’re just interested in the product of 2 committed values.

I’ll also add that as part of the statement being made, for which we need to make this argument for, the prover gives the verifier commitments for every a[i] and commitments to the rolling intermediate products b[i].

If we’re focusing on the case where n=3, we’ll have 2 bs, b[1] and b[2].

Proof Approach

The idea for this new scheme is that the prover will give the verifier commitments to constant values, create derived values a' and b' using verifier-sourced randomness x, ask the verifier to compute the product of the derived value a' with b', and have the verifier use x in an expression with the pre-committed constants to re-create the product of the derived values. Let’s start the explanation of this argument by diving into what the constants are.

What are the constants given at the beginning of the protocol?

The constants given to the verifier at the start of the argument are commitments to the same cross terms d[i] we described in part 1. These constants are given to the verifier before the verifier sends the prover their randomness x.

To make a product argument, we’ll eventually have to compute the product of tweaked values a' and b'. Since these values are tweaked, the tweaked product won’t only contain the public product we’re trying to prove as a term with an x coefficient, it’ll also include a bunch of extra terms. The coefficients of these extra terms can be defined before the verifier sends the prover x, and then later scaled by powers of x when the verifier re-creates the derived product.

If the committed coefficients do not describe the right polynomial structure, the prover is reduced to hoping the verifier’s random x still fits with their fake values. This is very unlikely, which makes this construct effective in preventing the prover from cheating. Just like in the previous product argument, allowing these constants to be chosen freely after seeing x can lead to a forged proof, so it’s imperative that the prover coughs up these values at the very start of the argument.

What are the derived values?

Once the cross term commitments have been sent by the prover, the verifier can now send them a random x term the prover can use to make derived values a' and b'. These derived values can be made using the following expressions: a' = sum(x^(i - 1) * a[i]) for i=1..n and b'=sum(x^(i - 1) * b[i]) for i=1..n

Proving the derivation uses a

As the verifier, the only term we know from the expression of the derived a' = sum(x^[i]*a[i]) is x. We don’t know any of the a[i] prover secrets. Since the verifier has given commitments to a[i] as part of the original statement, those commitments can now be used to link the secret preimages a[i]s to the public derived a'.

We do this by acknowledging that the commitment A' is computable using the commitments given to us at the start of the protocol, A[1], A[2], and A[3]. We then compute A[1] + x*A[2] + x^2*A[3] and do an equality check with A' to finish proving the link between the a[i]s and a'.

Proving we know the product of a[1], a[2], and a[3]

After proving the derived value a' is indeed derived from a[i], we need to show how the public product b can be derived from a' given a[1] and a[2] are both contained within the expression. In the previous product argument, we had a derivation for each term and had a way to link the product of the derived a[i]'s to the public product via reconstructing the commitment of the product between the derived a's. In this model, we’re just given a single sum of tweaks and have yet to find an operation on this sum that will yield the product of a[1], a[2], and a[3]. So the question is “what operation can we make on a' that will yield an expression with the product of a[1], a[2], and a[3] as a term?”

This is where the derived value b' comes in. We can multiply the derived sum a' with the sum of derived intermediate products b' to get an expression that suits our needs. If we think about the breakdown of the product from these derived sums, we see:

• a'*b' =
• (a[1] + x*a[2] + x^2*a[3]) * (b[1] + x*b[2]) =
• (a[1] + x*a[2] + x^2*a[3]) * (a[1] + x*a[1]*a[2]) =
• a[1]*b[1] + x*a[1]*b[2] + x*a[2]*b[1] + x^2*a[2]*b[2] + x^2*a[3]*b[1] + x^3*a[3]*b[2]

We can see the original product a[1]*a[2]*a[3] sitting right there as the coefficient to x^3, since a[3]*b[2] = a[3]*a[1]*a[2].

To verify the expression above, we then take the commitment of a'*b' and recreate it using the cross-term commitments sent at the beginning of the argument and other verifier-known values. For the case of n=2 (with the random term a[1]), the verifier wants to recreate commitments to the following terms that were coefficients to x^i in the expression above:

• a[1]*b[1]
• x*(a[1]*b[2] + a[2]*b[1])
• x^2*(a[2]*b[2] + a[3]*b[1])
• x^3*b

The constant value commitments mentioned earlier represent commitments to the extra coefficient terms in this derived product. By “extra coefficient terms”, I mean everything that’s not directly the final public product. Meanwhile, the final x^3*b term is computable by the verifier because b is public.

Running through the protocol end to end

Let’s recap this new protocol again to summarize the steps.

The prover starts by sending over the commitments to all possible cross terms between the secret intermediate values b[i] and the secret preimages a[i]. If n=3, the following represents all cross term products between a' and b':

• a[1]*b[1]
• (a[1]*b[2] + a[2]*b[1])
• (a[2]*b[2] + a[3]*b[1])

You might notice that included in these cross terms are some the intermediate products b[i]. For example, the a[2]*b[1] in the second term actually evaluates to b[2] This is because of the relationship between a[i] and b[i]: b[i]=b[i-1]*a[i] for i > 1

Now that the verifier has these values, they pick a random value x to give to the prover.

The prover responds with the derived values a' and b':

• a' will be defined as a' = a[1] + x*a[2] + x^2*a[3].
• b' will be defined as b' = b[1] + x*b[2].

It’s now the verifier’s job to multiply these values together and verify the constants we got at the beginning can be used to make the derived product a'*b' when plugged into our expected algebraic structure. Here, our expected algebraic structure is:

• a'*b'
• (a[1] + x*a[2] + x^2*a[3]) * (b[1] + x*b[2])
• a[1]*b[1] + x*a[1]*b[2] + x*a[2]*b[1] + x^2*a[2]*b[2] + x^2*a[3]*b[1] + x^3*a[3]*b[2]
• x^3*a[3]*b[2] + x^2*(a[2]*b[2] + a[3]*b[1]) + x*(a[1]*b[2] + a[2]*b[1]) + a[1]*b[1]

So we need to multiply:

• x^3 by the commitment for the final product b, since a[3]*b[2] = a[1]*a[2]*a[3]
• x^2 by the commitment for a[2]*b[2] + a[3]*b[1]
• x by the commitment for a[1]*b[2] + a[2]*b[1]

Then sum all of those values together with a[1]*b[1] to make sure it’s equal to a'*b'. And that’s it!

Optimizing away a term

There is one more optimization on the protocol we can make by changing the definition of b'. This alternate expression for b' uses the derivation b'=sum(x^(m - i - 1) * b[i]), with the exponent on the x counting down against the increasing indices of b. For our n=3 example, where we only have b[1] and b[2] as intermediate products, this means:

b' = x*b[1] + b[2]

So how does this help us?

In the paper I’m using as reference, ‘Efficient Zero-Knowledge Argument for Correctness of a Shuffle’, something called the zero argument is introduced. In the scalar version we’re using here, the zero argument can be understood as a cancellation trick. Aside from making the powers of x in b' count down instead of up, we also add one extra value to b side that equals the negative sum of the rest of the terms in b. This will cause one of the terms in the product a'*b' to equal the sum of intermediate products scaled by a power of x. Since the extra b value was chosen as the negative of the others, the coefficient on that x term now cancels to 0. This lets the verifier replace one reconstructed coefficient with a zero check.

This is easier to appreciate in the full vector argument because the zero argument is doing real work across many values at once. But even in this scalar walkthrough, the key idea is changing the direction of the powers lines terms up so that one part of the derived product can be compared to 0 instead of being reconstructed as another committed coefficient.

This is a follow-up post to the series I’m making about zero knowledge arguments. In this post I’ll be describing a way to prove the product of n hidden values using something called the ‘product argument’. I’ll focus on the case with just 2 hidden values at first, then inch my way closer to the reference method described in the Bayer–Groth single-value product argument.

I think this will serve as an interesting primitive to any zk-practitioner’s toolbox and is integral in understanding the Bulletproofs confidential transaction scheme that I’ll describe in a future post.

Before reading this, you should know that I take the liberty of making a commitment of a value mean the value is being multiplied by a generator G on the secp256k1 curve (e.g., commit(a) = a*G). In practice, we’d actually need something like a Pedersen commitment to make these schemes secure. Pedersen commitments add another randomness factor, and although I opted for simple non-Pedersen commitments, know that the high-level algebra doesn’t change much when this randomness is introduced.

What is the product argument?

The product argument is a protocol between a prover and a verifier where the prover aims to convince a verifier that the preimages of n committed values have a public product b. I’ll be focusing on the case where n=2 to introduce the subject.

Proof Approach

The product argument works by taking a similar proof approach to the sum argument. The idea is that the prover will give the verifier constant values, create derived values a1' and a2' using verifier-sourced randomness x, ask the verifier to compute the product of the derived values a1' and a2', and use x in an expression with our committed constants to re-create the product of derived values.

What are the derived values?

The formulas for the derived values are a1' = a1*x + d1 and a2' = a2*x + d2, where x is the public verifier-sourced randomness, and d1 and d2 are secret random values chosen by the prover.

The commitments to the values d1 and d2 (D1 and D2) are the constants given to the verifier before the verifier sends us their randomness x. This is to prevent the prover from cheating since letting the prover arbitrarily choose d1 and d2 after x has been revealed will allow them to forge a proof.

The operations performed on the original preimages to create the derived values can all be audited by the verifier, even without knowing d1 and d2. Let’s take a look at how this can be done by assuming the role of the verifier.

Proving the derivation uses a

As the verifier, the only term we know from the expression of the derived value a1' = a1*x + d1 is x. We don’t know a1 or d1, since those are prover-generated secrets, but we’re determined to check that a1' was actually derived from a1.

The trick to this is linking the commitment of a1 (A1 = a1*G) to the commitment of a1' (A1' = (x*a1 + d1)*G = x*A1 + D1).

Because of the algebraic properties of commitments, the commitment A1' is computable using A1, which comes from the prover’s original statement, and D1, which is one of the commitments given by the prover at the start of the interaction: commit(a1') == x*A1 + D1

The same process can be done for a2' using one of the other commitment constants, D2.

Proving we know the product of a1 and a2

After proving the derived values are indeed derived from a1 and a2, we need to show how the derived product a1'*a2' relates back to the original product of a1 and a2.

We can do this by showing the original product of a1 and a2 exists as a term in the derived product a1'*a2'. Let’s take a look at what the derived product of a1' and a2' look like when expressed in terms of a1 and a2 (liberally grouping the terms by their power of x): (a1*x + d1) * (a2*x + d2) = x^2*(a1*a2) + x*(a1*d2 + a2*d1) + (d1*d2)

We can see the original product a1*a2 sitting right there as the coefficient to x^2. To re-create this term, the verifier just needs to multiply the public product b with x^2.

The problem is the other two terms (the cross-terms). We can’t create the coefficient of the second term without (a1*d2 + a2*d1), or the 3rd term without (d1*d2). Since the as and ds are all secrets held by the prover, the prover can’t give these raw values to the verifier to check with.

Instead, the prover gives the commitments to these cross-terms, let’s call them E1 = commit(a1*d2 + a2*d1) and E0 = commit(d1*d2), in the beginning of the interaction as ‘constants’ to the verifier (along with D1 and D2). Verifying the equality by using the commitments of each term now becomes feasible: commit(a1'*a2') == x^2*commit(a1*a2) + x*E1 + E0

Why does the prover have to give these E1 and E0 commitments before x is chosen? Well, if the prover were allowed to wait until after they received x, they could just lie about the product. The problem would reduce to replacing the product commitment commit(a1*a2)in the following equation with an arbitrary value fake_product and solving for E0 like so:

Original equality:

commit(a1'*a2') == x^2*commit(a1*a2) + x*E1 + E0

Swap out a1*a2 with fake_product:

commit(a1'*a2') == x^2*commit(fake_product) + x*E1 + E0

Set E1 to any random value and solve for E0:

E0 = commit(a1'*a2') - x^2*commit(fake_product) - x*E1

Rolling products

Now let’s extend this idea to a rolling product. This means I want to prove b = a1 * a2 * a3 without revealing a1, a2, a3, OR the intermediate product of any chosen subset of those elements.

If I wanted to extend our concept to this, I don’t just need the commitments of a1, a2, and a3. I also need the commitment to the intermediate product, which we’ll call b_12 = a1*a2 (assuming I’m multiplying from left to right).

Then I could just perform the product argument twice! One argument proving b_12 = a1*a2 while keeping a1 and a2 hidden, and a second argument proving b = b_12*a3 while keeping a3 hidden. The only changes we’d have to make to our original product argument is having the prover give the verifier the commitment B_12 = commit(b_12) instead of a public scalar product b, and giving a commitment to b_12, b_12' = x*b_12 + d_b (where d_b is a new constant).

Taking this approach is much easier than doing a direct proof where we multiply all the derived values in one go, because then we’d be dealing with a higher degree polynomial and a nightmare of terms:

a1'*a2'*a3' = (a1*x + d1) * (a2*x + d2) * (a3*x + d3) =
x^3*(a1*a2*a3) + x^2*(a1*a3*d2 + a2*a3*d1 + a1*a2*d3) + x*(a3*d1*d2 + a1*d2*d3 + a2*d1*d3) + d1*d2*d3

I had trouble just typing that. Proving intermediate product commitments and keeping each argument to just 2 terms for the sake of simplicity just makes sense.

Rolling product optimizations

Let’s review the current rolling product process. As mentioned above, the prover creates a derived value for the intermediate product using a new blinder d_b: b_12' = b_12*x + d_b

With the straightforward product argument, the prover has to send the cross-term commitments (E1 and E0), the derived intermediate product b_12', AND B_12. I’d like to pose the question: can we get away with the prover never transmitting B_12? This would allow us to reduce the amount of data the prover sends the verifier.

Normally, if the prover provides the cross-terms but withholds B_12, the verifier would be stuck stuck. When the verifier would calculate commit(a1'*a2'), they’d have no way to check it, because the x^2 * B_12 could not be calculated with B_12.

But if the derived intermediate value is b_12' = b_12*x + d_b, that means its commitment is: commit(b_12') = x*B_12 + D_b

Notice that this expression contains B_12, and this can be used to our advantage to eliminate B_12 from the verifier’s check. If we multiply commit(b_12') by x, we get: x * commit(b_12') = x^2*B_12 + x*D_b

Now, let’s look at the verifier’s main equation for the multiplied derived values: commit(a1'*a2') = x^2*B_12 + x*E1 + E0

If we subtract the first equation (the original derived product expression) from the second equation (the expression with the intermediate value), the x^2 * B_12 terms completely cancel each other out: commit(a1'*a2') - x*commit(b_12') = (x^2*B_12 + x*E1 + E0) - (x^2*B_12 + x*D_b) = x*(E1 - D_b) + E0

The prover can simply combine their commitments and send (E1 - D_b) as a single cross-term, along with E0. The verifier computes commit(a1'*a2') - x*commit(b_12'), and checks if it equals x*(E1 - D_b) + E0.

Why does this work as a replacement to giving B_12 and checking to see if it’s in the original product and the intermediate value? Well let’s think about it from the angle of trying to cheat the verifier as the prover. If the prover lied about b_12' actually containing a1*a2, then the x^2 terms in the subtraction of the 2 equations would not match and could never be eliminated. Doing so would require the prover to commit to a value, that when multiplied by x, can magically get rid of the x^2 term and equal an expression made with prover-provided pre-committed values.

By verifying this way, the prover never has to send B_12 over the network, saving precious bytes in our proof size.

Conclusion

After going through the mechanics of the product argument, hopefully you see that it’s not too different from the sum argument. It’s a little bit more involved since the derived values x*a1 + d1 and x*a2 + d2 are polynomials, forcing us to do polynomial multiplication, but I believe the principles used in the argument are a natural extension to the principles used to build sum arguments. My next post will somewhat build on the concepts here to explain product arguments in the vectorized setting and get into how we can prove the inner-product of vectors in zero knowledge. Stay tuned!

This post builds intuition for the zero-knowledge arguments needed to understand modern range proof systems like Bulletproofs. It does this by introducing the sum argument, a simple technique for proving that hidden values add up to a known total without revealing said hidden values.

I decided to write this because, when I first encountered these schemes, I could follow the mechanics but struggled to convince myself they were actually secure. This post is my attempt to build that missing intuition.

The paper that finally helped things click for me was Efficient Zero-Knowledge Argument for Correctness of a Shuffle. I spent hours working through the same sections and trying to understand the ideas from different angles. The sum argument I’ll present here is not described explicitly in that paper, but it follows the some of the same underlying logic. The rest of this post focuses on explaining that argument as simply as possible.

What is the sum argument?

Let’s start with a definition. The sum argument is a protocol between a prover and a verifier where the prover shows that two committed values, A = commit(a) and B = commit(b), correspond to hidden values that add up to some known value c, without revealing what a and b are.

As a refresher, a commitment is the output of a function that is both hiding and binding. Hiding means it is hard to recover the input from the output. Binding means it is hard to find two different inputs that produce the same output. The concrete commitment function I will use is defined on the secp256k1 curve. It takes a scalar and maps it to a point on the curve, which we refer to as a commitment.

Proof approach

The goal is to show that c, or its commitment C = commit(c), can be constructed in two different ways using verifier-known values. These constructions need to be algebraically linked to the hidden values a and b in a way that would be very difficult to fake. At the same time, they cannot leak any information about a or b.

So the question becomes: how do we build expressions that are linked to a and b, but still keep them hidden?

A natural approach is to work with expressions derived from a and b. With that in mind, we want to:

• Create an expression a' that hides a, but still allows the verifier to check that it was actually used
• Create an expression b' that does the same for b
• Show the verifier how to combine a' and b' into a new expression c' that is linked to c
• Show a second way to construct c' using values that can be tied back to A and B

Proving we know a and b

To build expressions that hide a and b, we also need a way to convince the verifier that these expressions were actually derived from them. Since the verifier does not know a or b, they need to check this indirectly using the commitments A and B.

This is similar to proving knowledge of a private key when the verifier only knows the corresponding public key. A private-public key pair can be viewed as a preimage-commitment pair, so we can reuse that intuition here. In particular, we can use a signature as a derived expression.

This gives us a way to link expressions back to a and b through their commitments A and B, using well-understood techniques. To make this concrete, we’ll use the Schnorr signature scheme.

Schnorr signatures

A Schnorr signature consists of two components, R and s. Suppose a is the hidden value we want to build a derived expression from. The value s is defined as:

s = e*a + r

Here, e is a challenge chosen by the verifier (or derived from a hash), a is the secret, and r is a random value. The value R is the commitment to r, computed as R = r*G.

For our purposes, we will treat e as a verifier-chosen value so we can describe this in an interactive setting.

Using this construction, we can convince the verifier that we know the value a corresponding to the commitment A. The verifier checks this by computing S = s*G in two ways:

• Directly from s, by computing s*G
• From known values, by computing R + e*A

If both results match, then:

s*G = R + e*A

This works because s contains a inside it. When expanded, the equation becomes:

s*G = (e*a + r)*G = e*A + R

So any valid s must be consistent in being able to re-create an expression with the committed value A.

Now consider what happens if the prover does not know a and only knows A. Could they still produce values R and s that satisfy this equation?

If the prover could choose both R and s after seeing e, they could try to fake a valid pair by picking a random s and solving:

R = s*G - e*A

But in the protocol, the prover must send R before receiving e. This means R cannot depend on e. Once e is revealed, the prover must produce an s such that:

s*G = R + e*A

Finding such an s without knowing a would require solving a discrete logarithm problem, which is assumed to be hard. This is what gives the scheme its soundness.

Proving we know the sum of a and b

We now extend this idea to prove a relation between two hidden values.

To show that a + b = c without revealing a or b, we construct derived expressions for each value:

• a' = e*a + r1
• b' = e*b + r2

These follow the same structure as the Schnorr construction. As before, the verifier must choose e after receiving a commitment to the randomness. In this case, we commit to the sum of the random values:

R = (r1 + r2)*G

The prover sends R first, then receives e, and then sends a' and b'.

The verifier can now compute:

c' = a' + b'

Expanding this gives:

c' = e*a + r1 + e*b + r2

Grouping terms:

c' = e*(a + b) + (r1 + r2)

Since a + b = c, we can rewrite this as:

c' = e*c + r1 + r2

Now consider the commitment to c', which we call C'. The verifier can compute this as:

c'*G = e*C + R

Here, C = commit(c) and R = (r1 + r2)*G.

This gives the verifier two ways to create the derived value c' with known commitments. And since c' can be directly linked to c using the prover’s commitments, the verifier can be confident the expressions a' and b' truly contained the values a and b to make c.

Conclusion

Hopefully this post builds some intuition for how this proof approach can be generalized to other operations on committed values. In the next post, I’ll show how similar ideas can be used to prove products of committed scalars and even touch on inner-products with vector commitments.

It may seem abstract now, but these ideas are useful on their own and will come together when we get to Bulletproofs.

Ark

Before I get into the nitty gritty of the privacy properties of each flow, I’d like to give a basic overview of what the flows are. The 5 flows I’ll be covering are:

• onboards
• transfers
• balance checking
• offboards
• unilateral exits

As we go through the flows, an easy way to build intuition about what’s public or not is to note the distinction of on-chain vs. off-chain activity. When something is broadcast on-chain, everyone can see it. When something is computed off-chain, that information is known only to the involved counterparties until explicitly revealed to others. Now, let’s get started.

Onboards

Overview

Like all good layer 2 protocols, onboarding in Ark starts with a multisig transaction between the user and some counterparty. This multisig transaction acts like an escrow for which the ASP can claim a user’s funds once they’ve made a payment on the user’s behalf. In the general case, a user will send to their own Ark address when onboarding so they have layer 2 funds available for whenever they need it.

Looking deeper into this escrow transaction, note that the multisig is accomplished using a taproot output with a musig aggregated internal key gating the key path spend. In addition to this musig path, a script path is also included to allow one of the parties to claim their funds after a timeout. The party that gets to claim the timeout can differ between Ark implementations, but since I’m using Bark as my reference, it’ll be the server claiming the funds from this transaction if the time runs out.

To avoid having a bad ASP stealing our money by refusing to honor refunds from already-broadcast onboard transactions, we request a refund transaction from the ASP before the onboard transaction goes on-chain. We do this by asking the ASP to cosign a new refund transaction, using the unsigned board transaction’s output as input, that spends to a new script containing the refund path. I say ‘containing’ the refund path, because it also contains a musig and timeout path as well. This signed off-chain transaction will allow us to claw back our funds and is an example of a vtxo (virtual unspent transaction output), which is a utxo held off-chain that can be broadcast at any time.

Privacy

To start thinking about the privacy of onboards, let’s work through what an observer can learn from the on-chain transaction itself.

Assuming the board transaction is made with the Bark sdk, it’ll be a multi-input transaction with 1-2 outputs depending on whether we have change or not. The first output will always contain the amount we would like to board and is locked using a musig internal key along with a script that allows the ASP to take the funds after a timeout.

Since taproot hides both the internal key and the script attached to the output, an outside observer is none the wiser on whether the transaction is even an Ark transaction or even whether it has a script attached to it. Meanwhile, the ASP knows your inputs to the boarding transaction and the vtxo public key the refund spends to.

Now, let’s look at what a transfer looks like once we’ve onboarded a vtxo.

Transfers

Overview

Transfers can get tricky because there are 2 different types of transfers – in-round and out-of-round. When either kind is made, the sending user has notify the recipient via the ASP so the recipient can verify their new balance. This will be covered more extensively in the balance checking section. In this section, I’ll be covering more of the mechanics of the transfer itself.

In-round transfers

After onboarding and getting an initial vtxo, that vtxo is eligible for a transfer of either kind. When you make an in-round transaction, you are performing a swap between your vtxo and a new one that’s anchored to a different on-chain transaction. The way it works is you tell the ASP you want to participate in a round, they create the round transaction alongside another series of transactions that allows you to spend from that round, and assign you an output that’s locked by a hash.

You are then required to forfeit your existing vtxo in an offchain transaction, aptly called the ‘forfeit transaction,’ that’s different from the round you’re registering for. The output on this forfeit transaction is locked by a ‘sweep’ path requiring the ASP’s public key and that hash’s secret, along with a ‘refund’ path a user can use to reclaim the vtxo after a timeout. The idea is that the ASP would give you the preimage to spend your new vtxo and use that same preimage to sweep your old vtxo.

This hArk, or hashlock-based, approach is what superseded the connector-based approach featured in the original Ark protocol. It’s simpler to implement than connector chains but it does come with some privacy tradeoffs as we’ll see a little later when I touch on blinded output registrations.

Each round can have multiple senders and recipients, and is funded with the ASP’s own money. The series of transactions mentioned before that allow a recipient to put their vtxos on-chain use this round transaction as input to the first presigned transaction in the chain. Each transaction spends from one another and represent smaller round transactions with subsets of the original recipient group in their outputs. Each group can be broken down until there’s only 1 recipient, and the last transaction in this chain is what’s referred to as an exit transaction. Each recipient has access to the series of transactions that lead to their exit, and they can broadcast these at any time. It’s this exit transaction, which is initially held off chain like the rest of the transactions, that is signed over to the aforementioned hash in a forfeit transaction so it can be swapped for a vtxo in a new round.

Out-of-round transfers

Out of round transfers are those that don’t need an on-chain transaction to be made. Instead of new vtxos being anchored to on-chain transactions, they become anchored to the sender’s off-chain exit transaction. When a sender spends their exit to forfeit their vtxo, a new checkpoint transaction is made using the old transaction’s exit as its input. This can be conceptualized as an off-chain ‘round’ of sorts, except with one input and potentially multiple outputs that can be spent to an exit transaction for the associated recipient.

Privacy

For in-round transfers, the only on-chain/public action that’s produced is the round transaction. This round transaction, like other taproot transactions, has its internal key and locking script obfuscated from the world until spend time. Therefore, outside the participating parties, only the ASP knows about vtxo transfers. Unilateral exits can break that shared output with transactions that progressively dwindle the number of recipient groups, and reveal scripts in the process, but I’ll cover more of that in the unilateral exit section.

Balance checking

Overview

Every holder of Bitcoin on Bark needs to have a way to query the ASP for any vtxos sent to their Ark address. To prevent random observers from requesting the vtxos of an arbitrary address, an authorization system was built in the form of the Bark mailbox.

The Bark mailbox is a notification system where the ASP stores all notifications for a Bark wallet under a ‘mailbox identifier’ for that wallet. This mailbox identifier needs to be a public key for which the sdk makes a dedicated keypair for.

As mentioned before, a sender needs to give additional information to the ASP to push the notification to the receiver’s mailbox. This additional information can be found in the Ark address generated by the receiver’s wallet. At first, one might think this could be a huge loss for privacy if different addresses could be linked to the same mailbox public key, but Bark has preventative measures against this.

Instead of the raw mailbox public key being included in the address, a blinded/tweaked public key is provided instead. This blinded public key only needs to be decoded by 2 parties, the wallet and the server. So, the most straightforward way to do this is by tweaking the mailbox public key with a shared secret that can be computed using a Diffie-Hellman exchange. The input to this process for the receiving user is the private key belonging to the Ark address we’re generating the tweak for, and the server’s public key. Then when the sender tells the ASP they sent a vtxo to that address, the server can recompute the secret using its private key and the address the vtxo is assigned to.

Privacy

Because of this mailbox mechanism, an outside observer cannot learn about a Bark wallet’s balance unless given the proper authorization. There’s also no on-chain transaction that would allow an observer to learn much about the wallet’s mailbox public key (much like an on-chain transaction doesn’t reveal anything about the xpub of an on-chain wallet). The ASP of course knows the vtxos that belong to your wallet and the IP addresses you’re making requests from, similar to other consumer wallets.

Offboards

Overview

Offboards represent a collaborative exit path for vtxos in Bark. It involves telling the server which vtxos you want to offboard, getting an unsigned offboard transaction in response, signing it along with providing vtxos forfeit signatures, and receiving and broadcasting the final offboard transaction signed by both parties.

Unlike transfers, the forfeit transactions in offboards don’t use hashlocks to prove a transaction was made on their behalf, they use connectors. Connectors are transaction outputs first constructed from unsigned unbroadcast transactions, that are used as inputs to transactions that should only be valid if that unsigned transaction is successfully signed and broadcast. The Ark protocol was designed to use this mechanism for both transfers and offboards, but Bark switched to a hashlock exchange to minimize the on-chain footprint.

Privacy

Given that an offboard transaction can be funded by any of the ASP’s utxos, there is no direct link between the on-chain offboard transaction and the offboarding user’s vtxos. That means an outside observer does not learn which wallet requested the offboard.

Unilateral Exits

Overview

I already touched upon unilateral exits in the transfer section, but it refers to a way that a user can turn their vtxo into an on-chain utxo they can spend without any interaction from the server. When you perform a unilateral exit, a series of transactions are published on chain with the last one being freely spendable to an address of the user’s choosing. There are timelocks to allow an ASP to stop the exit of a vtxo that was forfeited to them, AND there’s a separate timelock that will let an ASP claim any exited vtxo after a period in a sweep, but once the user’s timelock ends (which is guaranteed to be before the sweep timelock ends) the user can spend it to wherever they want.

Privacy

From an outside perspective, still nothing much can be learned from this process. Ignoring generic transaction-graph leakage, the important privacy-relevant information is the revealed locking script and public key of the user who is exiting. If the unilateral exit was done on a sender’s already-spent vtxo, no information can be learned about the recipient that the vtxo was spent to. If the recipient were to perform a unilateral exit after the sender tried to exit their already-spent vtxo, an observer would then be able to link the sender and recipient by matching the hash revealed on both sides.

In summary, exiting a vtxo will create a new utxo that has an amount equaling the vtxo, but it still wouldn’t leak information on the wallet that created it. The sender of that vtxo can also identify the exit as having belonged to that recipient since it was linked to the hash they needed for their forfeit.

Blinded output registration

Another idea that has been floating around since Ark was conceptualized is this idea for it to operate like a coinjoin in making the ASP oblivious to which outputs a particular vtxo is spent to. In the connector-based design for Ark transfers, this worked, but in the hArk approach, things aren’t as simple.

When using connectors, a forfeit transaction’s spending condition is dependent on whether a transaction with a particular output set is broadcast on-chain. With the hArk approach, the forfeit transaction’s spending condition is dependent on an off-chain output being included in an already made transaction. A hArk forfeit itself is one side of an atomic swap with another vtxo.

This means there’s no ambiguity with respect to the output a forfeit transaction is attached to since both are linked by the hash in the new vtxo’s spending condition.

Final thoughts

I think the privacy afforded by Bark maintains a nice balance between privacy and usability. When used within the normal operating model, the ASP has no more insight into your transactions than the average wallet enabling on-chain transactions (so excluding wallets connected to one’s node). It’s also not too different from most consumer lightning wallets that use an LSP as the network gateway, although receivers can still have better privacy in lightning. All in all I think Bark seems to pay attention to privacy while not letting it bog down the user experience and I’m hopeful they’ll continue to uphold this standard.

In the previous post, we discussed pedersen commitments and how they hide transaction amounts while still allowing balances to add up correctly. We then touched on the topic of how one can fake 2 outputs that cancel out to spend more money than they have. Range proofs are the patch for this. They let you prove that committed values stay within sane bounds without revealing the values themselves. Without range proofs, confidential transactions don’t work.

The high level details

The flavor of range proofs we’re using is borromean range proofs. Borromean range proofs bring in a new cryptographic construct called ring signatures. Ring signatures let a signer within a group of public keys sign on behalf of the group without exposing themself as the signer. If you like analogies, you can think of this as a police lineup: the verifier knows the real signer is in the lineup, but can’t tell which individual it was. To motivate its use within a range proof, let’s take a look at a somewhat naive solution.

Naive approach

Imagine we have a list of pedersen commitments where each commitment is a curve point C generated like so: C = vG + bH

What’s cool about curve points is that any point can be treated as a public key in secp256k1, even C. To sign for C I would need to know its discrete log with respect to G, which means I’d need to find a value c such that C = cG. Unfortunately I don’t have a scalar c in C = cG so I can’t actually sign for C.

I can sign for the public keys derived from the terms of C though since I know the private keys are v and b respectively. We’ll use this to our advantage later on. Now let’s use this concept to make a rudimentary range proof where we’ll prove v is in the range of 1 to 2.

One way I can prove that the value v in my pedersen commitment is either ‘1’ or ‘2’ is to make a scheme with ring signatures using public keys where all of the public keys are derived from C.

Each public key is generated by taking C and subtracting a value pG where p is represents a possible value we’re trying to prove our value v might assume. So if I’m trying to prove v in C = v*G + b*H is either 1 or 2, I will make 2 public keys C - 1*G and C - 2*G.

What does this scheme do for us?

If I’m not lying about my value being 1 or 2, one of my derived public keys will subtract the correct value, allowing me to sign with the pure H multiple b*H.

For example, if my value v is 1, then the first term in the list of public keys would evaluate to bH. This is because C = 1*G + bH and if we replace C in C - (1*G), we get (1*G + bH) - 1*G = bH.

Meanwhile I could never sign for the public key X = C - 2*G because I don’t know the discrete log for the resulting point X. 2*G won’t cancel out one of the terms I know, and I’d need to solve the elliptic curve discrete log problem to find x in x*G = X.

Since we’re utilizing a ring signature scheme, I can sign for all the public keys as a group and not reveal to an outside observer which public key subtracted the correct value.

Thinking more broadly, we can use this strategy for an arbitrary number of values from 1 to n, instead of 1 to 2 like in our toy example. The downside to this scheme is that it requires additional data for every possible value added. The bigger the range, the bigger the signature size grows. This led researchers to the question of whether the size for this scheme could be shrunk down and, lucky for us, the answer was yes.

Binary decomposition

The path to reducing the data of the naive scheme in the last section is through arithmetic. Researchers figured the difficulty of mathematically identifying the signer in a ring signature doesn’t get reduced if we reduce the number of partipants. So lets do that.

Imagine we have a ring signature for a naive range proof where we’re trying to prove a pedersen commitment C contains a value v (in C = v*G + b*H), where v is either 0 or 1. The public keys for this would be C - 0*G or C - 1*G.

Now imagine we made a second ring for a pedersen commitment C2 where the possible values are 0 or 2. A cool way to link these 2 pedersen commitments, C and C2, is to think about the range of the sum of the vs in these 2 commitments. Since the first pedersen commitment has v = 0 or 1, and the second pedersen commitment has v = 0 or 2, the lowest the sum of these values (aka the v terms in the pedersen commitments) can be is 0, and the highest the sum of these values can be is 3.

Now think of the case where we can make each ring signature be a range proof for 0 or a power of 2. Then when we can break apart a pedersen commitment with a large value v into new pedersen commitments that have new vs that sum to the old v. Just to reiterate, the additional requirement is that the new commitments can only have v be a power of 2.

Now you can think of each ring signature as bits that can commit to a power of 2 being ‘on’ or ‘off’. For example, say we were signing a range proof proving a value in a pedersen commitment was between 0 and 3. In this case, we sign for the ‘on’ value of ring signatures that contain a v with a power of 2 present in the binary representation of 3. So instead of one ring proving “v is one of {0,1,2,3}”, we’re using 2 to prove the 2^0 bit is 0 or 1 and the 2^1 bit is 0 or 1.

We will also have to split b, but b doesn’t have to get split into its binary representation. We just need to make sure we split b up into n parts, where n is the number of rings we would like to use (n is log_2(v) because it also represent the number of bits needed to represent v). These bs should actually be chosen somewhat randomly with the method I previously described when choosing blinding coefficients for commitments on a transaction’s outputs (in part 1). We can make all numbers random except the last one, and the last number can be chosen to make the sum equal b.

For intuition about this multi-ring approach, we’re saving space because the number of values we’re able to prove exponentially scales with the number of rings (just like adding a bit to series of bits lets us represent twice the amount of numbers). The key observation is that it’s easier to represent bits than exhaustively represent every number.

With separate independent ring signatures, someone could mix-and-match valid signatures from different signing sessions. Borromean ring signatures bind the rings together so they’re all bound to a single signing event. With separate independent ring signatures, someone could mix-and-match valid signatures from different signing sessions (akin to vulnerabilities to CBC-style encryption).

The requirements are knowledge of the elliptic curve discrete log problem and elliptic curve field operations.

Why should you care about privacy on the blockchain?

I want you to think about every financial transaction made within the past year. Political donations, pharmacy purchases, food delivery orders, etc. Now imagine all that information made public. Or worse—made available to a blackmailer to abuse. Every piece of information about you in the wrong hands can be used against you. Information about what you’re purchasing, who you’re purchasing from, or your purchasing amount is no exception, yet this is the case we find ourselves in with most blockchains today. So what do we do about this privacy problem?

Addressing the privacy problem

Having good privacy hygiene and not reusing addresses does some of the heavy lifting in pseudonymizing recipients, but one fundamental component to privacy is being able to hide transaction amounts. Even if I can’t see what you’re purchasing, knowing that you spent $500 at a pharmacy tells me plenty. At first, hiding amounts on the blockchain may seem like an impossible task, but there is a way! This is what’s accomplished by confidential transactions.

What are confidential transactions?

Confidential transactions are transactions where the output amounts are hidden to everyone but me, my recipient, and anyone we decide to share it with. The scheme takes the amount I’m spending and ‘hides’ it with a function that allows everyone to verify I’m not spending more money than I’ve received. The output of this ‘hiding’ function is what we refer to as a commitment, and given a commitment, there is no feasible way to recover the original input amount. The original amount is instead hidden in a different part of the transaction for the recipient to recover so they can verify it corresponds with the commitment. If the amount didn’t correspond with the commitment, no future recipients would accept the transfer.

In this post I’ll be focusing on the function that transforms the amount into a commitment, which in this context is a “Pedersen Commitment.”

What are Pedersen commitments?

Let’s break this term down. The first thing I’ll explain is what a commitment is. A commitment is an output value of a function f that takes in a secret, or multiple secrets, as input. What makes it a commitment is the property that the input can’t easily be derived from the output (which we refer to as hiding), and that another input can’t easily be found that creates that same output (what we call binding).

Pedersen commitments rely on the assumption that the ‘discrete log problem’ is hard. With this assumption, it’s not computationally feasible to find another input that equals that output. Opening a commitment means obtaining the input from the committer that produces the commitment.

The paper that introduced Pedersen commitments, written by Torben Pryds Pedersen, used it as part of a larger secret sharing scheme and originally formed the commitment like so:

g^a · h^b mod p

Where a represents the number we want to ‘commit’ to, and b represents a per-commitment nonce, called the blinding factor, that makes sure we can make different commitments for the same committed value.

In the context of elliptic curves, the expression becomes:

a*G + b*H

The original scheme worked in a multiplicative group where g^a and h^b are both integers for which multiplication is defined. For elliptic curves, a*G and b*H are points for which point multiplication is not defined—we only have point addition. This naturally causes a change in the expression to use point addition instead.

How do Pedersen commitments help us hide transaction amounts?

The main goal of designing a confidential transaction scheme is to maintain the integrity of the blockchain so that no money is unjustly created. Pedersen commitments allow us to do that by allowing us to perform arithmetic operations on commitments.

Because I can perform arithmetic operations on commitments, I can subtract the sum of all output commitments from the sum of all input commitments and verify that the result is 0 to verify no money is being created (or destroyed in a way that’s unaccounted for). In short, using f to represent the function that turns values into Pedersen commitments, our check looks like:

f(inputs) - f(outputs) == 0

Let’s try an example. Imagine we have one input value of 3 represented by the commitment in_0 = (3*G) + (7*H). With this input we make 2 outputs—one with the value of 2 represented by the commitment out_0 = (2*G) + (4*H), and another with the value of 1 represented by the commitment out_1 = (1*G) + (3*H). An observer to our transaction is not able to break our commitments into the additive terms, but they can perform the following comparison to show the inputs are equal to the outputs:

in_0 - (out_0 + out_1) == 0

Inserting our example values for this, we get:

(3*G) + (7*H) - ((2*G) + (4*H)) - ((1*G) + (3*H)) == 0

If we move terms so that all the Gs and Hs are grouped together (since addition is commutative on elliptic curves) we can reframe the expression as:

(3*G) - (2*G) - (1*G) + (7*H) - (4*H) - (3*H)

Both the amounts (3 = 2 + 1) and the blinding factors (7 = 4 + 3) cancel out independently, which is what makes this work.

Notice that the output blinding factor isn’t completely random since both the input and output coefficients of H need to cancel out to 0. To minimize the disruption of this restriction, we usually pick all the output commitment blinding factors except the last one randomly, and pick whatever value cancels out the sum of the other blinding factors as the last one. So basically:

last_output_blind = sum_of_input_blinds - sum_of_other_random_output_blinds

Unfortunately we have a big problem with our scheme. Namely, the problem of ‘negative’ valued outputs. I say ‘negative’ in quotes because there are no negative values on elliptic curves in the traditional sense. Instead, we refer to a number a as negative with respect to another number b if it is large enough to make the sum a + b wrap around the modulus and produce a result less than b. See my last blog post for more information about “negative” numbers on elliptic curves. For now, let’s keep conceptualizing these numbers as negative.

So here’s the problem. Imagine I have 1 input value of 4 and 2 output values of 1 and 3 respectively. Let’s also say I make commitments for them:

in_0  = 4*G + 2*H
out_0 = 1*G + 1*H
out_1 = 3*G + 1*H

Let’s think like an attacker here. What’s stopping me from adding an extra output back to myself for a value of 30? If I did, subtracting output commitments from input commitments would now equal -30. Well how about if I added another output with a value of -30 (where -30 is modulus - 30)? These 2 outputs would cancel out in the input/output equality check and no one would know because the values are hidden.

Addressing this problem necessitates the addition of a new tool in our confidential transaction toolbox which we’ll refer to as range proofs. Range proofs are a way to prove that all outputs are within a range of values that won’t cause an overflow, and will be the topic of my next blog post “Confidential Transactions for Dummies Part 2.”

Prerequisite knowledge

The only requirements are algebra and modular arithmetic.

Elliptic curves

Elliptic curves are what we call functions of the form y² = x³ + ax + b. These functions can be written in different ways, but the representation I’m using is in “Weierstrass” form. The curve used for Bitcoin-related things, secp256k1, looks like this in Weierstrass form:

y² = x³ + 7

And if we graph it, it looks like someone punched a vertical line from the right:

It has a nice shape to it though—like the Snapchat logo turned on its side:

Elliptic curves on finite fields

Well, it was too nice for cryptographers, so they had to make the x & y values wrap around with a modulus! For secp256k1, this modulus is prime. I’d like to say it’s chosen to be prime because it sounds cool (since mathematically you can now say the underlying field form A GROUP OF PRIME ORDER!!!), but there are reasons for the selection that I won’t get into here.

Anyway, this is what it looks like with a modulus applied:

With a modulus, we’re able to wrangle an infinite number of points into something finite and countable. And since mathematicians love to count things, as I’ve learned through the math classes I struggled through years ago, we know how many points are in a modded elliptic curve as well. Let’s label the modulus applied to the coordinates of elliptic curve points “p”, and the number of points this function can generate “n”. In secp256k1, n is slightly less than p.

One REALLY important thing I need to cover before moving forward is that for cryptographic applications, we always mod the numbers we pass into finite field elliptic curve functions. You might be thinking “obviously! you told us they’d be modded by p,” but we actually don’t use p as the modulus for scalar inputs to our function. We use n! “Inputs” here refers to the scalar multipliers we use in elliptic curve operations, not the resulting point coordinates. I just had to mention that to avoid confusion moving forward.

Elliptic curve point operations

There are also operations we can perform on points, like adding one point to another, but the algorithm to do that is non-obvious and the details are somewhat irrelevant here. It is important to know it can be done in this special world of elliptic curves.

Why do we mod scalars by ‘n’ instead of ‘p’??

After n, the scalar multipliers we input start to produce the same points again since mathematicians proved that the points on the curve make a cyclic group. One of the properties of this cyclic group makes it so we have n unique points on the curve! Well, why not just work with scalars mod p instead? Because the scalar multiplication operation has a period of n, not p—meaning after n steps, we return to our starting point.

I’ll also mention that after modding the scalar by n, all coordinate calculations are done mod p. Remember, n is the order of the curve (number of points), while p is the field size for coordinates.

The curious case of negative numbers

Coming back to the title, the TRUTH about negative numbers in finite elliptic curves is that they don’t exist in the traditional sense! When I talk about ‘numbers’ here, I’m referring to what cryptographers call scalars—the multipliers used in elliptic curve operations. So not point coordinates.

For the elliptic curves we’re discussing, I mentioned scalars are mod n (the order of the curve), and they’re defined to be non-negative integers from 0 to n-1.

So where do negative numbers come from? Am I duping the readers by definition?? Well, yes and no. I’m using “negative” here to mean “the additive inverse”—the thing you add to get back to zero.

With the set of integers, every element has an additive inverse (i.e., 2 + (-2) = 0). With the set of integers modulo some number n, every number still has an additive inverse, but it’s just the number that, when added, gives you zero mod n (i.e., 6 + 1 ≡ 0 (mod 7)).

With elliptic curves we work with prime p as modulus, which also makes p odd. What does this have to do with anything? Well, I can say that an even number added to another even number will never equal our odd p!

We can then label even numbers as ‘positive’ and odd numbers as ‘negative’ since every element in this set of positive numbers has an inverse in the odd set of numbers. There will never be an inverse of an even number in the even set or vice-versa, making the inverses mutually exclusive. It’s this mutual exclusivity that lets us relabel these sets as ‘positive’ and ‘negative’. ‘Negative’ is therefore redefined in this context to mean a set of inverses to another set rather than the traditional concept of negative.