Breaking Changes coming in Testnet

The binary format that we use in Move (bytecodes and binary tables; as defined in libra/language/vm/src/file_format.rs) is going through some changes in the next Testnet release (week of 1/13/2020) and in the coming weeks.
We are updating the format by adding/modifying instructions and correcting/updating some of the binary tables.
We will communicate the exact changes in the release notes, but we wanted to give you a heads up to ensure people can get ready if they need to apply changes.

Who is affected by those changes?

If you are developing within the Libra code base you should not see any problems and everything should update as you recompile.
If you are developing and deploying a client and you have compiled whitelisted transactions, those changes will affect you. The transactions will have to be recompiled or regenerated given that their hashes will change.
Testnet will only accept a set of whitelisted transactions, those transactions are hashed and the hash is used to verify whether a transaction is known. By changing the binary format we are changing the content of those transaction and thus the hashes. So, if one has an old transaction and submits it, that will be rejected. Again, the transaction changes hash because the script in the transaction will change and so the content of the transaction is different.
If you are using the Libra codebase to generate the script/transaction you should be able to continue to do so and get the proper updated transactions.
If not, you need to regenerate the script according to the new binary format, which is defined in libra/language/vm/src/file_format.rs.

Please let us know how we can help if you are having difficulties or issues in getting Testnet to accept your transactions.

2 Likes

Thanks for letting us know, it was addressed somewhere else already, but as it fits here now very well: are there any plans to have different kind of Testnets, or some kind of “resilience” to updates? There are some clients out there already, mostly created, supported and maintained by people doing this voluntarily and it’s hard to keep up with the pace of the “master” development. But if a client currently fails to keep up, the only fallback is a “private” Testnet.
There is some room for improvement here, especially if Libra wants to establish a vital ecosphere of 3rd party libs and applications. My first guess would be to either let companies to this for reputation or let the consortium provide different Testnet versions (at least “current/master” and “stable”).

1 Like

This is definitely something to keep in mind though, as you can imagine, it is a pretty hard problem and decision. I’ll bring it up internally and see if others have plans for that.
I am curious though how you see that happen. Having a testnet for each breaking change may be too much and it could just fracture the ecosystem even more. Trying to leverage some upgrade/versioning story/protocol could be an idea once that is defined, but that also may not cover some of the changes coming.
Any thoughts on what that could be?

1 Like

I can only suggest general workarounds, as I am not too much into the (Rust) part of development.

In general these things could ease the pain a lot without changing the way you develop:

  • Provide an easy way of creating a private Testnet (with Docker) with a minting possibility besides the original client
    • Create a simple testsuite for gRPC calls (at least minting/transactions/query balance,events), so developers with “clientless” mode can develop easily with this test suite
    • Keep this test suite with the source code to have actually running test cases against each version

If it’s possible to change the dev process, I’d suggest

  • Separate breaking changes from non-breaking changes for alternating releases
  • Maintain at least 2 and at max 3 different branches (2 stable, 1 dev), those are Testnet candidates
  • Define 4 likely dates (end of each quarter?) which will most likely bring breaking changes

Improve documentation process

  • For breaking changes, less prose and more pseudo-code would be great
  • Sample of “before” and “after” requests etc. could make implementing changes easier
  • Please keep in mind that though Rust is the new Black, there are many devs out there not able to read it fluently, so pseudo code is great

Change the responsibilities

  • Libra (Foundation)/Calibra or whatever it is should not be responsible for all, dev and ops and brainstorming and spec asf
  • With good incentivization 3rd parties or consortium members could take over certain tasks
    • Provide resources, hardware and humans
  • Incentivization could be reputation by listing companies on the libra.org site
  • Grants could be given to 3rd parties like Gitcoin etc. in Ethereum

Hope this helps, TL;DR: there are 20+ members in the Libra Foundation, some of them really big. Why are they not participating with this? There could be a Calibra Testnet, but also an Uber Testnet, a Lyft Testnet etc. Same for all other tasks…

2 Likes

This is something we’ve considered and might to do in the future in order make it less volatile to develop on top of Libra. Before Libra launches, however, changes to APIs and formats are to be expected.

Do mean something like what’s in here? https://github.com/libra/libra/tree/master/docker

Great idea!

I think you’d have to ask them, but running a testnet isn’t the only way to participate :wink:

1 Like

It would be great to have the possibility to run the validator locally from docker but those are not currently working and I have never got them to work. I guess they are meant to be run in the aws environment as the start fails by referencing some configuration files that don’t exist in the project.

To me, that’s a good starter, but it’s very Rust client focused - eg. I am missing a guide to just start the validator and then instructions on how to use Libra “client-less”, eg. by gRPC calls samples or a thin client in a more common language than Rust. In general, the “ecosphere” (documentation, etc…) of Libra is not language-agnostic enough.

But it’s quite cheap for a big win in reputation - maybe it would help to clarify this - and having an incentivization scheme would be even better.

Personally, an incentive scheme would be a good idea to motivate. :+1:

Re participation and incentivization for Consortium members or external parties, I create this list some time ago: Developer Onboarding

1 Like