Share the municipalization models with everyone, please
Published in the Boulder Daily Camera, 3/24/13
Boulder City Council should not view the municipalization of our electrical supply system as "a game to be won." Rather, municipalization should be seen as a decision to be made in a way that best serves Boulder's citizens. After watching events unfold over the past few weeks, it looks like the city may suddenly be more interested in winning than in necessarily making the best decision.
What caught my attention was Boulder's refusal to provide the city's municipalization models to Xcel. Over the past year, city staff and their outside experts have developed several options for municipalization that they translated into computer models. These models predict the extent to which each municipalization option meets future electrical demand cleanly, reliably, and cheaply. As such, Boulder's case for municipalization is embedded within these computer models.
The optimistic model predictions that were presented to City Council last month reflected only the inputs from the city's model development team. The model development team admitted that the model relied heavily on hundreds of their assumptions and guesses about the future, which is par for the course when building complex models like these. However, the thing you want most after completing AnyBigModel, Version 1.0 is fresh eyeballs scrutinizing your work. And, if you want models that predict the future accurately, the eyeballs you want most are from the skeptics asking hard questions about your model, not those of acolytes who like the answers your model is giving.
When Boulder decided not to let Xcel review their model, they broke that fundamental rule of computer modeling. Boulder turned away the expertise of Xcel, who knows more than anyone about whether the assumptions and data in Boulder's models are valid.
Boulder's rationale for keeping the models from Xcel was that they may be relevant in a future lawsuit with Xcel. While legally defensible, this argument against giving the models to Xcel makes little sense. I can think of little in the computer model that would help Xcel in any lawsuit that is not already available in the 287-page model report available from the city's website.
Alternately, the potential benefits of allowing Xcel to support Boulder by identifying weaknesses in Boulder's analysis are huge. At a minimum, Xcel's review will initiate a public dialogue about the assumptions made by the city. However, it's also possible that Xcel could find genuine flaws that would render invalid the model predictions that city staff presented to Council last month. Boulder residents will understand Xcel's biases and view anything they say with appropriate skepticism. However, we can only be wiser by hearing Xcel's viewpoint.
The city staff went on to suggest that Xcel could use the input and output data contained in their modeling report to rebuild the model while, at the same time, making clear that Boulder had no intention on giving Xcel the actual formulas that were programmed into the model. I reviewed the report and, in my opinion, this statement is profoundly misleading. Before hitting the editorial pages of the Camera, I earned a Ph.D. in Systems Engineering and had a 30-year career in computer modeling science and technology. From this background, I can comfortably say that the devil in any computer model is in the details. By withholding mathematical formulations, Boulder is hiding from Xcel the logic that is central to how the model works and, by inference, the predictions that the model makes. Xcel needs the model, not just the report, to adequately understand and critique Boulder's work.
The other more onerous explanation for Boulder withholding the models is that the city fears that the models have weaknesses and they would prefer that these soft spots not be exposed to public scrutiny. Whoever else looks at these models, no one will have the expertise nor resources of Xcel and, if they find issues, Xcel will certainly bring them to the public's attention.
If the city's goal is to make the best decision, then Xcel's expertise and input can only help them make a better decision. On the other hand, if the city has so much invested in municipalization that it has become a game to win, then keeping the models away from Xcel is the best move the city can make.
The City of Boulder has every right to municipalize, but a decision of this magnitude should only be made with a bright light shining on the people making the decision and their reasoning. Keeping the models from Xcel is just a way of dimming that light.
Boulder City Council should not view the municipalization of our electrical supply system as "a game to be won." Rather, municipalization should be seen as a decision to be made in a way that best serves Boulder's citizens. After watching events unfold over the past few weeks, it looks like the city may suddenly be more interested in winning than in necessarily making the best decision.
What caught my attention was Boulder's refusal to provide the city's municipalization models to Xcel. Over the past year, city staff and their outside experts have developed several options for municipalization that they translated into computer models. These models predict the extent to which each municipalization option meets future electrical demand cleanly, reliably, and cheaply. As such, Boulder's case for municipalization is embedded within these computer models.
The optimistic model predictions that were presented to City Council last month reflected only the inputs from the city's model development team. The model development team admitted that the model relied heavily on hundreds of their assumptions and guesses about the future, which is par for the course when building complex models like these. However, the thing you want most after completing AnyBigModel, Version 1.0 is fresh eyeballs scrutinizing your work. And, if you want models that predict the future accurately, the eyeballs you want most are from the skeptics asking hard questions about your model, not those of acolytes who like the answers your model is giving.
When Boulder decided not to let Xcel review their model, they broke that fundamental rule of computer modeling. Boulder turned away the expertise of Xcel, who knows more than anyone about whether the assumptions and data in Boulder's models are valid.
Boulder's rationale for keeping the models from Xcel was that they may be relevant in a future lawsuit with Xcel. While legally defensible, this argument against giving the models to Xcel makes little sense. I can think of little in the computer model that would help Xcel in any lawsuit that is not already available in the 287-page model report available from the city's website.
Alternately, the potential benefits of allowing Xcel to support Boulder by identifying weaknesses in Boulder's analysis are huge. At a minimum, Xcel's review will initiate a public dialogue about the assumptions made by the city. However, it's also possible that Xcel could find genuine flaws that would render invalid the model predictions that city staff presented to Council last month. Boulder residents will understand Xcel's biases and view anything they say with appropriate skepticism. However, we can only be wiser by hearing Xcel's viewpoint.
The city staff went on to suggest that Xcel could use the input and output data contained in their modeling report to rebuild the model while, at the same time, making clear that Boulder had no intention on giving Xcel the actual formulas that were programmed into the model. I reviewed the report and, in my opinion, this statement is profoundly misleading. Before hitting the editorial pages of the Camera, I earned a Ph.D. in Systems Engineering and had a 30-year career in computer modeling science and technology. From this background, I can comfortably say that the devil in any computer model is in the details. By withholding mathematical formulations, Boulder is hiding from Xcel the logic that is central to how the model works and, by inference, the predictions that the model makes. Xcel needs the model, not just the report, to adequately understand and critique Boulder's work.
The other more onerous explanation for Boulder withholding the models is that the city fears that the models have weaknesses and they would prefer that these soft spots not be exposed to public scrutiny. Whoever else looks at these models, no one will have the expertise nor resources of Xcel and, if they find issues, Xcel will certainly bring them to the public's attention.
If the city's goal is to make the best decision, then Xcel's expertise and input can only help them make a better decision. On the other hand, if the city has so much invested in municipalization that it has become a game to win, then keeping the models away from Xcel is the best move the city can make.
The City of Boulder has every right to municipalize, but a decision of this magnitude should only be made with a bright light shining on the people making the decision and their reasoning. Keeping the models from Xcel is just a way of dimming that light.
