Neural Networks (NN) are recognized as universal approximators, yet the reasons behind their remarkable generalization capabilities remain partially unresolved. One potential explanation is the manifold hypothesis, which suggests that NNs identify low-dimensional manifolds within the high-dimensional real-world data. In each layer of a NN, the data lies on/near a manifold, a so-called representation manifold, which gets progressively flatter as the layer's depth increases. Since NNs function on these data manifolds rather than isolated points, generalization becomes feasible. Notably, manifolds in the deeper layers of trained NNs tend to be flatter, while at the same time, deeper networks generally achieve higher accuracy. This leads to the following hypothesis: There is a correlation between the accuracy of a NN and the flatness of manifolds in the final layer of the NN. To investigate this, we empirically test the hypothesis by training a CNN with various hyperparameter sets. We assess both the flatness of the manifold and the accuracy on a test set, exploring their connection. We use correlation analysis, plots, statistical tests and examine whether flatness can be predicted from the hyperparameters.