I suspect that Musk was told that though he could be on the board, he'd basically be wasting his time because nobody there was receptive to things like free speech and consistently applying the rules.
Tesla CEO Elon Musk will not join Twitter's board - CNN
"As part of his agreement to join the board, the Musk had committed to not acquiring more than 14.9% of the company’s shares during his term. Some corporate governance experts suggested that the move could have been a way of reining in Musk’s influence over the company. Now, that limitation is no longer in place, leaving the door open for Musk to potentially take a more aggressive stance by buying up more of Twitter’s shares."