Network Security Proposal - Cancel and Revert Validator Upgrades to v1.0.5

Governance & Proposals
1

Disclosure
For this proposal I am not just speaking as a community member, investor, and owner of the Nova validator on the Terra Classic chain, but also as a senior engineer who has had experience in the industry for over 15 years, has programmed in a multitude of languages such as Rust, Elixir, Swift, Golang, Solidity, Javascript, and more; as well as built and worked on distributed systems in a multitude of industries including the crypto space in both consultant and entrepreneur based roles. It is from that body of experience that I am drawing upon when I put forth my reasoning for the creation of this proposal in the first place. I speak as both an industry professional and a validator who is responsible for helping to secure the Terra Classic chain, which is a role I take very seriously.

MOTIVATION & AIMS:
With the recent passing of Proposal 11310, validators have been preparing in the background to begin the upgrade process to v1.0.5. However, with the recent news that has happened as of late in regards to Allnodes, compromised validators, and the overall security of the Terra Classic chain and the entire Cosmos network itself, it has become clear that the proposed timing for validators to upgrade to v1.0.5 by block height 11,543,150 (approximately February 14th, 2023) needs to be reconsidered and postponed so that the v1.0.5 release itself can be looked at it in more detail and more thorough testing can be done.

The reasoning for wanting to do this are many, and from a professional development perspective many yellow and red flags have emerged since last week that make validators postponing the upgrade the only safe choice we can currently be taking to ensure that we are fulfilling our role of properly securing the Terra Classic blockchain. Those flags are as follows:

  • PFC’s departure from L1 JTF in under a month (yellow flag)
  • L1 JTF’s failure to properly communicate about the details of the v1.0.5 release in the Terra Classic Validator discord in a timely manner (yellow flag instead of a red flag considering everything that’s been going on)
  • Jacob Gadikian’s departure from L1 JTF in under a month due to the Allnodes security risk (red flag)
  • Zaradar’s public handling of the Allnodes security risk and denial of it for being the risk it is (red flag)
  • Notional’s departure from the chain, as well as other notable validators starting to consider their departure from the chain as a result of the L1 Team’s handling of the Allnodes security risk (red flag)
  • The fact that there needed to be an updated v1.0.5 Upgrade release that changed it from being a non state breaking change to a state breaking change, which is an indication of inadequate testing (red flag)
  • The fact that the upgrade seems to currently not account for 7TB of state data that already exists on the blockchain, which may be intentional but needs to be confirmed all the same and could also indicate a potential loss of smart contract data (red flag)
  • The fact that after much talk about implementation of Constrained Pools, not only is the code poorly written from a professional perspective via it’s use of:
  • But also the fact that Zaradar proceeded to merge the contentious changes for his Constraint Pools (which are implemented incorrectly) into the main branch when it hasn’t passed code review from other developers on the team anyways (red flag): Constrained pool to ensure LUNC can never have more then 2T supply by ZaradarBH · Pull Request #51 · classic-terra/core · GitHub
  • The actions of which then forced another developer (inon-man) to create a second pull request to revert the changes that Zaradar should have never merged into main in the first place, and giving excuses upon inon-man calling him out for said behaviour that goes against how development on teams is intended to work, and also having another developer (lehajam) have to pipe in and explain that Zaradar’s code was fundamentally incorrect and have to explain to him why it was even broken in the first place (which is generally fine in practice, but is an issue when Zaradar has merged the code into main without the changes having gone through proper code review - a very HUGE red flag): Revert "Constrained pool to ensure LUNC can never have more then 2T supply" by inon-man · Pull Request #82 · classic-terra/core · GitHub

All of the above reasons are very concerning, but any developer who has been paying attention to the codebase like I have has also already observed these behaviours from Zaradar. These behaviours and all the other yellow and red flags that have surrounded the release of v1.0.5 that when combined with the fact that a large amount of the voting power on chain currently consists of validators that have had their seed phrases compromised by Allnodes’ business practices creates a multitude of vectors of risk that make it impossible for any sane validator that is working to secure the Terra Classic blockchain to adopt the v1.0.5 release at this time until our chain security issues have been more thoroughly handled and the actual code that is present in the v1.0.5 release has been more deeply looked at, and more testing has been done (outside of just the L1 Task Force doing the testing) then what we’ve seen to date.

WE PROPOSE:
That all validators hold off on upgrading to the v1.0.5 release on their machines until chain security has been sufficiently improved to a point where the chain is no longer compromised or at risk. We’ve already got enough of a crisis on our hand with the chain’s security being compromised, the last thing we need is to risk potentially losing chain state. Once chain security is back at a point of confidence, we can look at releasing a new version of v1.0.5 that has an updated upgrade block height and has had it’s code thoroughly reviewed by multiple validators and experts in Cosmos like Jacob Gadikian.

By voting YES - You acknowledge that there is risk to the chain present in v1.0.5, and that validators should NOT upgrade their nodes to v1.0.5 so that the code can be more closely looked at by validators and undergo more thorough testing at a later date when the chain’s security isn’t in such a tenuous state.

By voting NO - You do not think that there is any risk to the chain present with v1.0.5, and that validators should not do their due diligence on the code of the release and push back the upgrade, and that they instead should upgrade their nodes anyways despite the many red flags and risks presented above.

By voting ABSTAIN - You have no particularly strong thoughts on the matter either way.

By voting NO WITH VETO - You are vehemently against the entire premise of this proposal.

Signed,
Bilbo Baggins

3 Likes
3

I’ve been feeling good about my investment until now, after reading the evidence exposed in this proposal am starting to feel a little worried for my bag. After taking a closer look though, I can see that it’s got some solid facts THAT YOU CAN’T DENY, z-lovers.

4

Who are inon-man and lehajam are they on l1 task force

1 Like
5

You have a lot of dev experience. Do you have any thoughts of applying as a security officer for the L1 TF team? It would be better to persuade in the team than to advise from outside.

7

Oh yeah delay v1.0.5 so we can’t get v1.0.6 and wallet whitelist for Binance before 1st March, then have Binance drop LUNC support with no more burns and watch LUNC crash. Good plan to attack LUNC. NO with veto.

3 Likes
8

No thank you.

9

Have you checked, what was changed in this specific release?

10

In your haste to create proposals, have you had time to ask for comments and explanations from L1 TF regarding this?

4 Likes
11

that appears to be the story… attack attack… participate… no thanks.

1 Like
12

Of course not. The plot & agenda of him and the group of people around him is clear as day.

1 Like
13

At this point, I honestly believe that if His Holiness The Pope farts, it’ll be viewed as an adverse reaction to JGs removal/withdrawal from the L1TF team.
This whole situation is now a joke and should not be viewed in any serious light.
Go in peace. But please go … and stay gone! Thank You!

2 Likes
14

My response to this prop is based on zero years of coding experience and complete lack of technical knowledge as it relates to this network upgrade. However I am certified in spotting Bull Shit props and this one has 10 Red Flags.

Nearly all of your arguments are completely unrelated to the code up grades. The majority of them have personal opinion bias, attacks on contributing members, and motive based on business facets that don’t favor certain parties within the community. Non of this prop speaks to any concrete, undeniable reason why we should not do the upgrade on a technical level or states .

Hard NO on this prop. It lacks supportive evidence of upgrade risk and majority of the contexts is politically motivated with numerous mentions of information that could be subjective and interpreted in differed ways base on a thesis that has nothing to do with the upgrade.

6 Likes
15

@StrathCole
Of course I’ve reviewed it. The concerns in specific are around the upgrade handler and the testing that has been done around it, like I’ve mentioned in this proposal. The conduct of the L1 JTF’s programming leadership (particularly Zaradar) has only added to my concerns around the integrity of this release, for the reasons I’ve listed above.

@Tonu_Magi
Here. You can see my posts as Nova Validator engaging with the L1 TF (particularly Ed) multiple times about the upgrade.

It’s worth noting that Ed’s response here only happened as a result of me following up with him in the Terra Classic Validator Discord, which you may not be a member of since it tends to be focused around validator specific duties.

image
image2902Ă—1210 371 KB

Note also this post from Jacob Gadikian pointing out his concerns as well around the upgrade handler, which is a concern that has yet to be addressed as well. Updated v1.0.5 upgrade - #34 by faddat

@SockPuppet
@lunaticlunclover69
@EZMehrtenz
@Lunc_Global
Put aside any dislike you may currently have for how Jacob handled the Allnodes security risk situation.
Put aside any dislike you may have for me for putting up proposals that the community disagrees with because they’re unpopular opinions that are based on facts that can be tangibly pointed to.

I have a lot to lose as a validator if this upgrade handler blows up in a way we don’t expect. Every validator does. If we end up losing state on the chain because we didn’t do the proper testing and due diligence on this release as a group of validators, that’s a big problem that we can’t really easily come back from and we’ll have failed our delegators and we’ll have failed to secure the chain.

Postponing the release so everyone can take a closer look at it and give it the proper testing it deserves, and also giving ourselves time to sort through the fact that our chain is filled with compromised validators right now, is the responsible thing for every validator to be doing.

I honestly don’t see why this is even up for debate.

16

I think you don’t want to help.If the 1.0.5 update is delayed, the support decreases, there is no point in being a validator.
We have to stick to the roadmap. We can resolve this issue later. I think the prioritization is not correct.your approaches are starting to get boring.
We trust the L1 team. We don’t want to change. We don’t want an alternative L1 power. If these are not what you want, you can be a validator in another chain.We need the L2 team, contribute to the L1 team and improvements. Approaches such as do not develop, do not work are not correct.We don’t want another L1 team. You can leave if you want.

2 Likes
17

So you linked the convo with Ed and Ed said how they will handle it.

Then instead of proposing a better solution or inquiring further on the matter - you made a proposal that doesn’t offer a solution. Just asks for more time to investigate it.

I mean - isn’t your time better spent actually working on a PR to offer up?

I’m sure you have made many contributions so far, yes?

2 Likes
18

@Tonu_Magi
There is no part of that conversation that describes handling the issue at hand. You are getting your facts wrong yet again Don. Putting up a proposal to delay adoption of the release, investigate the codebase and take the time necessary to test it is the solution.

It’s called a “CancelUpgradeProposal” in Cosmos (read: x/upgrade | Cosmos SDK), but since our codebase is so out of date, we don’t have that as an option. So the next best thing is an Agora post.

I also don’t see why you feel the need to keep trying to question my contributions when there’s compromised validators that are refusing to remake their validators and validator wallets and an L1 JTF that is unilaterally merging in broken commits on a whim that other developers then have to play defence on. Those are things I’ve been handling that are bigger priorities and you know that because I’ve told you about that and shown you the proof of that as well.

This isn’t about me “helping out” by making pull requests. This is about testing and code review and verification of a release that has a potential risk to it if things are done wrong that can result in the loss of chain state. They’re entirely different things.

Also @tarikmandal I can tell you right now no sane validator should care about any L1 Team’s internal roadmap if it’s putting the chain at risk. Our job is to secure the chain. Not to ensure the L1 Team meets their roadmap.

This is development 101, guys.

19

If you read the convo between JG and Ed - you’ll see what was their solution to this.

So, unless you actually have alternative and viable solutions at the given time, with given tools - I suggest you let the L1 TF do their work, since that is what they are being paid for.

Now, as to your concern to compromised validators - how does this tie into 1.0.5 release? Is it relevant to 1.0.5? Or is it just another attempt at changing the subject.

Proposal isn’t about what PFC did or why Notional left or what Zaradar believes.
Yet you like to sprinkle in these tidbits as somehow pertaining to the code.

And as to the code - Ed said what he discovered and how they plan to overcome the issue.

Mayhaps be part of the solution for once?

2 Likes
20

Hello there,

This is basically why I left. I did not want to be pressured to approve of code that wasn’t good.

I I think that there are simply better ways to achieve the aims.

21

Don, Please link to the conversation between Jacob and Ed where they described a solution to the problem of upgrade handlers not accounting for real world chain state.

Read my reply to you again Don. I was addressing a point you made to me. You’re shifting arguments again while accusing me of being the one doing it.

Every point I’ve made is all part of a larger pattern that goes back months that leads to a very alarming situation to be in. Don, where there’s smoke, there’s a fire.

Please show me where you saw that.

This proposal is the solution that we have at this given time, with the given tools (your own words). You are just refusing to acknowledge that it is.

22

Your proposal lacks any form of a solution.

Who will be overseeing and approval of “proper” code?
Who will test it?
What will be the approximate timeframe?
What are the associated costs?

Drawing attention to a larger issue, while containing the proposal to one specific version upgrade?

Read Eds response. I’m on mobile. Hardly a welcoming environment for proper formating, but verbatim Ed said they will trigger it a specific block height (for 2.0.4) and that is their solution.

Now, as we discussed back in December - your team of Terra Allies was supposedly looking into Upgrade Handlers and you wanted to get a proper grasp on the subject. You and your team?

You identified an issue, waited after the proposal got passed, waited some more. Proposal is hastily written to stop and…?
What exactly are you solving by this? Who are you serving by this?

2 Likes