Questions To Ask Your Cosigners
Validators are guardians of cryptographic keys. On Penumbra, a validator uses a variety of cryptographic keys which control its identity, governance voting, consensus signing, and self-delegation wallet. All of these share a sobering truth: theft or loss of these keys can have devastating consequences. On behalf of their delegators, validators like Starling Cybernetics are tasked with the responsibility to keep our keys secure, yet available for legitimate use.
I want to show you how you can guard your keys and your funds with the same security measures used by validators like Starling Cybernetics, using threshold multi-signature custody with decentralized key generation (DKG).1
All the core technology you need to use threshold custody with Penumbra is already built in to its pcli
command-line wallet – but great technology on its own is not enough to achieve great security. That's why I also want to share with you some questions you can ask yourself and your collaborators to help establish a baseline of operational norms for threshold custody. I used this set of questions as I created the threshold custody setup for Starling Cybernetics, and I believe they're helpful – feel free to improve upon them, and let me know your thoughts.
This post will cover:
- Background about threshold custody and decentralized key generation,
- Considerations when selecting a threshold committee,
- Techniques for securing your computing environment, and finally
- The questions themselves which I sent to my own cosigners.
Let's begin!
Threshold custody
One of the most secure ways to store a valuable cryptographic key is K-of-N threshold multi-signature custody. This method of key storage fragments a key into N shards, each of which is stored by a separate device, person, or organization. The underlying key can only be used when a predefined threshold quorum K of these cosigners collaborate to use it, usually to authorize a transaction.
Threshold custody is resilient to theft and loss
Confidentiality and availability are typically in tension, but threshold custody can be both secure and available. If a key is stored by engraving it on a steel plate and storing it in a bank vault, it can't quickly be accessed when the time comes to use it. On the other hand, if a key is stored on your laptop, it's convenient to use it, but it's vulnerable to theft or loss. By contrast, a key stored using K-of-N threshold custody is:
- Resilient to theft of up to (K – 1) of N shards, since the thief would still not have access to a threshold quorum K of N; and
- Resilient to loss of up to (N – K) of N shards, since there would still remain the threshold quorum K of N shards with which to sign.
By way of concrete example, a key stored using 3-of-5 threshold custody requires a quorum of 3 of 5 cosigners, so is resilient to theft of 3 – 1 = 2 shards, and resilient to loss of 5 – 3 = 2 shards.2
Threshold custody is more private than a multisig
In a traditional multisig custody setup, a smart contract holds funds and only authorizes a transaction when it receives signatures from a certain quorum number of custodians. On its face, this sounds very similar to a threshold custody setup – but it is crucially different, because multisig wallets are visible to the world, since they exist on-chain. Consequentially, anyone can learn details about the multisig: how many participants there are, how many of them need to sign a transaction, and sometimes even information about their identities! This property decreases the overall security of a multisig wallet, because it increases the attack surface of its participants.
By contrast, a threshold custody setup as implemented on Penumbra is publicly indistinguishable from any other custody method: the public cannot tell whether threshold custody is even in use at all. By keeping the mere existence of the setup completely off-chain and private, threshold custody on Penumbra protects cosigners – and with them, your funds.
Avoiding single points of compromise
As we've seen so far, once shards are distributed to cosigners, threshold custody can be a highly secure way to store valuable key material. But what happens before the cosigners receive their shards?
If the shards are created by a so-called “trusted dealer” who generates a master key and splits it into shards, this requires trust – not merely in the good intentions of the dealer, but also in the security of the device on which the shards were generated, and the communication medium used to transmit them to the cosigners. For a moment in time, even if brief, all the shards of the key are united when the key is born – in the moment of its birth, the whole of the key could be compromised in one fell swoop.
Luckily, we can do much better:
Decentralized key generation (DKG)
Far more secure than using a trusted dealer is to use a decentralized key generation (DKG) ceremony, where the key is collectively generated by the participation of all the cosigners. In the ceremony, each participant runs the same software, which instructs them to send a sequence of messages to other participants, in a particular sequence. In the end, each participant ends up with their own specific shard of a newborn key. Using a DKG gives mathematical certainty that even if they tried to, no cosigner can ever see more than their shard of the key, even for an instant.3
A threshold DKG custody setup with distinct human cosigners can form the basis of extremely secure storage for sensitive and valuable key material4 – and the pcli
command line wallet supports threshold custody with DKG for all Penumbra users. What's more, thanks to some code I wrote, Penumbra validators can also use threshold custody for all their key material – not just for the wallet holding their self-delegated funds, but also for their identity keys and separate governance keys.
As is appropriate, Starling Cybernetics uses threshold custody created using DKG ceremonies for all high-value key material. This technique safeguards our identity key, our self-delegation wallet, and all other significant assets we store on Penumbra.5
Selecting a threshold committee
It's a serious task to participate on a threshold committee: one must be trustworthy, discreet, resistant to coercion, technically and operationally competent, and available when required. When selecting members of your own threshold committee, pick people for whom you have strong reason to believe that they embody these traits, even under exceptionally bad circumstances. Remember that if K of your N cosigners collude without you, then they can act together without you!
You may also want to consider the diversity of your cosigners. The more different from one another they are, the less likely it is that a single threat could compromise multiple of them. Factors to consider when evaluating diversity for cosigners might include: where they live (geographically and politically), who they know (socially and professionally), what their financial incentives are, and more.
It is a good idea to keep utterly private the identity of the cosigners on your threshold committee. As we have seen above, it is possible to conceal from the world all the details of your threshold custody setup, in contrast to the forced disclosure of traditional multisig custody. Take advantage of this! If nobody knows how many people or which people control your shards, then nobody can try to seek them out. What's more, coordinating to create or use the threshold key only requires communication between you and each cosigner in the quorum, so cosigners need not even know how to contact one another. This internal anonymity can act as a barrier against collusion, although you should have a very high degree of trust in all your cosigners regardless.
Securing your computing environment
In a threshold custody setup with N > 1, the theft of a single shard is not sufficient to compromise the key. However, if several participants use their everyday insecure computing environment to handle their key material, the risk of multiple shards being compromised increases.
Because of this, it's a good idea to plan for your cosigners to use computing environments that are resistant to targeted hacking and spear-phishing. Even if no such attacks ever occur, you can sleep easier at night knowing that your shards have been handled with care.
Two good options for secure computing environments are the operating systems Tails and Qubes. Tails is a lightweight "amnesiac" operating system which can be booted from a flash drive and leaves no trace of your activity on the computer hosting it, or, by default, even on the flash drive itself. Qubes is an installable operating system with heavier and more specific hardware requirements than Tails, which uses separate virtual machines to compartmentalize data so that even if one piece of the machine is compromised, it doesn't affect other activity. Another possible option, though more difficult to use, is a total physical airgap between a secure machine and the world: provision it with the needed software, and then cut off its network access forever, carefully using storage media to move data to and from it.
Although no security solution is perfect, handling sensitive key material using an environment like this can significantly raise the bar for any adversary trying to compromise your shards, because most attacks on your cosigners' day-to-day computing environments should not have any effect on the secure environment they use for custody operations. It's worth doing your own research to determine which secure computing environment(s) are best-suited for your needs and your cosigners' abilities. In advance of the DKG ceremony, to make sure that there won't be serious technical difficulties, remember to test all the software you plan to use for the ceremony in the environments each of you select.
The questions
The security and availability of a threshold custody setup starts at its beginning. I'd like to share with you the survey I sent to each cosigner on the threshold committee that safeguards some of the highest-value keys for Starling Cybernetics, in advance of our DKG ceremony.
These questions are meant to establish a baseline to understand the capabilities and risks of your cosigners, as well as to help you clearly communicate your expectations to them, so you can empower your cosigners with the tools they need to help keep you and your funds safe.
Computing environment
Once you've done your research and talked it over with your cosigners, consider asking them:
- What secure computing environment will you be using?
- Do you want help setting up your environment?
- Does your cell plan permit hotspot tethering? Tethering to a hotspot rather than using your home or office internet can make it more difficult for a potential hacker to access the device, because even if they compromise your local network, it won't help them get to the secure device.
- Do you have a flash drive which can be temporarily used for the ceremony? This is likely only required if you are using a live flash-drive environment like Tails.
- How many free USB ports are there on your device? If you are using a live flash-drive like Tails, you will likely need 2 or more, so that you can load the operating system from one flash drive and store generated key material on another.
- Do you need any other physical supplies for the ceremony itself which you don't yet have? How can I help?
Storage capabilities
This survey is tailored around the assumption that each participant should keep a primary copy and backup copy of their shard, each encrypted, in physically separate secure locations. You should decide on your own set of procedural requirements and recommendations that you are comfortable with, and clearly communicate that expectation to your cosigners so they know what you'd like them to do.
This subset of questions is meant to explore the capabilities your cosigners can provide for secure physical storage. Depending on how you decide they should store their shards, only some of these might be needed:
- Do you have a dedicated flash drive on which you could store your shard?
- Do have a printer?
- Do have a CD burner and CDs?
- Do you have tamper-evident bags?
- Do you have a safe deposit box at a bank?
- Do you have access to any other secure storage locations?
Primary storage
For most setups, the primary copy of the key material should be in a location that is easy to access on a timescale of a few days, for at least K of the N cosigners.
One reasonable storage method for the primary copy could be on an encrypted flash drive which is used solely for that purpose, and which is stored in a secure location when not in use. When considering how to store this key shard, keep in mind the diversity of events that might compromise its confidentiality or integrity: natural disasters, insider threats, etc.
Consider asking your cosigners:
- What storage media will you use for your primary copy of the key shard?
- Where, physically, will you store your primary copy of the key shard? You may want to avoid knowing the precise location, but answers like "in a bank vault" or other secure undisclosed locations can be helpful.
- Will anyone else other than you have physical access to the location of the primary copy of the key shard?
- What other physical security methods are in place to protect the shard?
Backup storage
It's a good idea to back up key material in at least one other physically separate secure location besides its primary storage location. This location does not usually need to be accessible on the same timescale as the primary key.
One reasonable storage method for the backup copy would be to encrypt it and print it out on a piece of paper, stored in a separate secure location from the primary copy. The advantage of a paper copy is that it's immediately evident whether paper still holds information, whereas an electronic storage device can fail without your knowledge until it comes time to restore the backup. Regardless of your backup method, you'll likely want to protect backups from physical damage in the first place: consider using protective supplies like sealed fireproof bags.
Consider asking your cosigners:
- What storage media will you use for your backup copy of the key shard?
- Where, physically, will you store your backup copy of the key shard? You may want to avoid knowing the precise location, but answers like "in a bank vault" or other secure undisclosed locations can be helpful.
- Will anyone else other than you have physical access to the location of the backup copy of the key shard?
- What other physical security methods are in place to protect the shard?
Encryption
Both the primary and backup key material should always be stored encrypted, regardless of location or storage medium.
One good option for this is the age
encryption program, which is compatible with both encrypted paper backups and Yubikey hardware key custody (though not both at once). Many other file-encryption programs exist that could serve this purpose: it's worth doing some comparative research to determine which best fits your needs.
Once you've talked through the possibilities with your cosigners, consider asking them:
- What program will you use to encrypt the primary copy of your shard?
- What program will you use to encrypt the backup copy of your shard?
- How do you intend to store the encryption key(s) (e.g. passphrase) for your shard?
- If using a password manager to store the encryption key, how is access authenticated? Some password managers have additional security options like 2-factor authentication of various kinds.
- If using a hardware security device or paper to store the encryption key, how and where will it be stored? This should be distinct from the location of either shard.
- Is this encryption key storage method compatible with the secure computing environment you will be using? Make sure all the software to create and restore your primary and backup shard can run in that environment.
- Do you need any physical supplies for long-term storage which you do not yet have? How can I help?
Confidentiality
The fewer people who know the identity of your cosigners, the better. You probably want to know:
- Can you be overheard or overseen in the space where you would be performing the initial ceremony and subsequent signing?
- Who else could reasonably come to know that you are the custodian of these shards? What is the nature of the relationship with these people?
Continuity
It's hard to think about bad things happening to people you like and trust, but it's important to plan for the unlikely event that something happens to one of your cosigners – or to yourself.
To understand these scenarios, consider asking:
- If your primary means of access to the encryption key for your shards fails, do you have another way of accessing it? Hardware can break, data can be corrupted or deleted, and human memory is fallible.
- If anything were to happen to you, would anyone else ever be able to recover your shard? Depending on your trust assumptions and threat model, "yes" or "no" might be the more desirable answer here. Consider communicating how you feel about this to your cosigners.
You may also want to instruct your cosigners to execute instructions on your behalf in the event that something happens to you – for example, to communicate with a trusted person in your life.
Communication
You will likely want two distinct communication methods with your cosigners:
- A way to coordinate a signing request with your cosigners: I suggest the Signal messenger, but there are other reasonable choices here.
- A way to exchange the automated messages required to perform the DKG and sign transactions: I suggest Magic Wormhole, which creates an encrypted tunnel between two computers authenticated by a random passphrase you can share through the former secure channel.
It's important to set clear expectations for your cosigners about communication. A threshold custody setup means that no matter how urgent the need, you cannot act without their help – every cosigner should know what level of availability is required of the role.
Additionally, you might establish a set protocol for contacting your cosigners to request that they use their shards. When developing this protocol, consider how to prevent someone from successfully impersonating you, especially in the presence of rapidly-advancing deep fake technology.
To establish expectations about communication, you could ask:
- In general, how quickly should I expect to be able to reach you with a request to sign a transaction?
- After receiving a request to sign a transaction, how quickly could you access your key shard?
- When you receive communication from me, how will you verify my identity?
- Please list all the ways I can contact you in an emergency.
- Please provide at least one emergency contact.
Lastly, perhaps the most important question of all:
✨ When are you free for the DKG? 🔑🔑🔑
Stay safe out there,
—finch
P.S. Thank you to Swan, Henry de Valence, and @erwanor for insightful and detailed editorial commentary on this post pre-publication.
Want to catch the next post?
Any suggestions or recommendations are meant as a jumping-off point for your own independent planning to determine solutions that fit your specific needs. Starling Cybernetics, and I personally, do not assume liability for the consequences of your actions: do your own research.↩
The mathematically astute may notice that theft-resilience and loss-resilience are equal to the same number of shards for any K-of-(2K – 1) threshold, i.e. 2-of-3, 3-of-5, etc., because (2K – 1) – K = K – 1. This is even true of a 1-of-1 threshold (i.e. a singular full key): this lonely custody setup is resilient to the theft or loss of 0 of 1 shards.↩
The fact that DKG is possible, let alone practical, still surprises and delights me – it feels far from obvious on its face that mathematics should allow us such a useful ability.↩
Based on what I know, it seems possible that a threshold custody setup with a threshold greater than 3 of K has never been compromised in real-world practice – see this Twitter thread by @hdevalence.↩
Furthermore, we use the excellent Horcrux to make our consensus key secure and highly available by means of threshold custody amongst a committee of machines.↩