Always Should You Locally Generate and Sign Your Transaction — Here is Why

Introduction

To transfer coins on a blockchain, you usually build a transaction, sign it using your private key, and submit the signed transaction to the blockchain network for execution. During the building of a transaction, a JSON transaction is first populated, and then s/erialized to a raw transaction (BCS bytes), on which the subsequent signing process will work.

In some situations, you may be incapable of serializing a JSON transaction locally. To overcome the incapability, a few blockchains provide API endpoints that help you serialize the JSON transaction remotely. This seems to be a convenient feature for those unable to do the serialization on their own. However, this feature could also be a huge risk and cause the loss of all your funds from the security perspective.

In this blog article, I will tell why you should NEVER depend on these remote API endpoints, but should ALWAYS generate and sign your transaction locally.

Risk Details

I would like to take the Aptos blockchain as an example.

In the Aptos blockchain, there are plenty of ways that you can send a transfer transaction. You can either use aptos CLI, or utilize several available SDKs to accomplish the task.

For security assessment purposes, I investigated the internal implementation of these methods, and found out that there exists a huge risk when using the official Python SDK to transfer.

As shown by the official example , you can call the RestClient.transfer method in Aptos Python SDK to send a transfer transaction. Inside this method, a transaction request txn_request is made, submitted to the REST API endpoint /transactions/encode_submission to produce a raw transaction to_sign, which will then be signed by the user’s private key, and sent to the Aptos blockchain network for execution.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

# https://github.com/aptos-labs/aptos-core/blob/7e7deefa689aafac6e44959c8d4738c1dbebe180/ecosystem/python/sdk/aptos_sdk/client.py#L111-L148
    def submit_transaction(self, sender: Account, payload: Dict[str, Any]) -> str:
        """
        1) Generates a transaction request
        2) submits that to produce a raw transaction
        3) signs the raw transaction
        4) submits the signed transaction
        """

        txn_request = {
            "sender": f"{sender.address()}",
            "sequence_number": str(self.account_sequence_number(sender.address())),
            "max_gas_amount": "10000",
            "gas_unit_price": "100",
            "expiration_timestamp_secs": str(int(time.time()) + 600),
            "payload": payload,
        }

        response = self.client.post(
            f"{self.base_url}/transactions/encode_submission", json=txn_request
        )
        if response.status_code >= 400:
            raise ApiError(response.text, response.status_code)

        to_sign = bytes.fromhex(response.json()[2:])
        signature = sender.sign(to_sign)
        txn_request["signature"] = {
            "type": "ed25519_signature",
            "public_key": f"{sender.public_key()}",
            "signature": f"{signature}",
        }

        headers = {"Content-Type": "application/json"}
        response = self.client.post(
            f"{self.base_url}/transactions", headers=headers, json=txn_request
        )
        if response.status_code >= 400:
            raise ApiError(response.text, response.status_code)
        return response.json()["hash"]

The problem is that the response from the REST API service provider is trusted unconditionally. You will sign on for the unreliable response data and post the valid signature to the REST API service provider.

In fact, this provides one way for the REST API service provider to get you to sign anything that it wants, which enables it to empersonate you on the Aptos blockchain!

Note that I also found a similar issue in the Sui blockchain, where its javascript SDK depends on the remote RPC service for transaction serialization by default.

Reference: https://github.com/MystenLabs/sui/issues/6606

Attack Scenario

Think about that you connect to a REST API service provider, which is corrupted.

Every time you send an EncodeSubmissionRequest to the REST API service provider, it will replace the should-be raw transaction with that of a malicious transaction as the response (for instance, a transaction that transfers all the coins of your account to an attacker’s account).

Upon receiving the response, you will automatically sign on, without any warning, and submit the original transaction together with the signature of the malicious signature to the REST API service provider.

Now, this transaction is meant to fail passing the signature verification. However, the signature of the malicious transaction is acquired by the REST API service provider, and it can rebuild the malicious transaction, attach the valid signature to it, and publish the constructed malicious transaction to the blockchain network.

The signature of the malicious transaction is valid! The malicious transaction would soon be executed by the blockchain, and as a result, all funds of your account would be lost!

Likelihood of an Attack

Since the profit of amounting such an attack could be far beyond the benefits of providing the REST API service, there is a high chance that a third-party REST API service provider or a private one would exploit such a vulnerability and do something evil.

In theory, there is also a slight possibility that the official might use this vulnerability as a backdoor to manipulate some on-chain activities.

Remediation && Recommendation

For blockchain developers:

• Well document the risk of using such a remote API for serialization, and warn users of the risk if they are using this feature.
• Support more ways for users to serialize locally.
• Disable, if necessary, this remote serialization functionality.

For blockchain users:

• Seldom, or never, use the remote API for transaction serialization, do it locally instead.
• Only if the upstream node is completely trusted should you depend on a remote API for transaction serialization.

Conclusion

The lesson is that never trust any input from outside. Every unreliable input from outside should be validated carefully, and must be discarded if not validated or unable to be validated.

The transfer transaction is one of the most important components in the blockchain world, and thus should be secured with extreme effort. Both developers and users should be responsible for the secure usage of the transfer transaction.

These issues were reported on Dec 2022.

It’s worth noting that the Sui blockchain responsed agilely, acknowledging this issue and soon deprecating these remote API.

Unfortunately, the Aptos blockchain didn’t consider this feature as something dangerous. As of the time of writing, the Aptos Python SDK still uses the remote API to serialize the JSON transaction…