Over the past two years, there has been rapid and unprecedented progress in data-driven, AI-based models for weather prediction. The paper evaluates the forecast quality of Pangu-Weather, a state-of-the-art AI weather model, and two physics-based NWP models: the ECMWF ensemble, and MEPS, a high-resolution limited area model. The forecasts are compared based on temperature and wind speed data from observation stations in Norway. Overall, the authors find that the unprocessed forecasts of the MEPS model are superior to those of the ECMWF ensemble and the Pangu-Weather model (which perform similarly). After post-processing, the relative differences in terms of the evaluation metrics are much smaller, with slight advantages for the post-processed MEPS forecasts.

In my view, the paper is timely and addresses interesting and important research questions given the recent rise in data-driven weather forecasting. The two main contributions are that 

• the paper provides (potentially the first, but at least one of the first) comparisons of AI-based and physics-based weather forecasting models based on station data (rather than the commonly used comparisons based on gridded ERA5 data);
• the paper assesses and quantifies the effect of post-processing on forecasts from AI-based weather models. The paper is well written; the findings and conclusions are presented clearly and are supported by the presented results. Some minor comments, questions and suggestions are summarized below. 

General comments:

1. To me, the most interesting contribution was on the effect of post-processing on the AI-based vs. physics-based weather models. Though not entirely unexpected, I found it interesting to see that after post-processing, the differences between the ECMWF and Pangu-Weather forecasts seems to be minimal. This contribution of the paper might be strengthened by highlighting it more, for example in the ‘Conclusions’ section.
2. Regarding the first comment from above, it might have been interesting to add some notion of ‘significance’ or uncertainty to the observed score differences, in particular after post-processing. I would expect that (potentially except for parts of the MEPS scores) the differences likely are not significant. 
3. In terms of post-processing methods, only BQN is applied. Even though I assume that this is not the case, it might be interesting to discuss whether there are any reasons to assume whether different effects of post-processing could potentially be expected for other post-processing approaches.
4. The BQN post-processed forecasts only use weather variables directly related to the target variable. However, it has been demonstrated in various papers (on physics-based weather models) that neural network-based post-processing methods such as BQN can benefit substantially from including additional predictors. Would you expect similar improvements when utilizing additional predictors from AI-based weather models?
5. In the presentation of the results in Figures 2 and 3 (and 4 and 5), I found it somewhat confusing that the line types do not consistently refer to raw and post-processed forecasts. Personally, I would have found it more instructive, if (for example), scores for deterministic forecasts are always drawn in solid lines, and forecasts for probabilistic scores in dashed lines (or vice versa).

This manuscript is very well written. I like its length and clarity, and I believe it adds further evidence that ML/AI will revolutionize weather prediction. I have one minor technical comment (below); otherwise, I think the manuscript is well done and ready for full publication.

Page 3: Add a (ME) after mean error (second line from bottom) for consistency with other metric definitions and Table 1.

You conclude that the models, including Pangu-Weather, are considerably less skillful than the high-resolution MEPS model.

a) Does this make a strong case for a high-resolution/limited area re-analysis dataset?

b) Along those same lines, given a high resolution re-analysis dataset to train on, would you expect that ML/AI models will be equally competitive or better than traditional high-resolution NWP models and ensembles, and that the ML/AI approach will be equally skillful at predicting higher resolution, higher-impact meteorological phenomena? Or would high-impact meteorological prediction be better served by additional post-processing of the current resolution of Pangu-Weather and other global ML/AI models?