It isn't just “private ownership of the means of production”. Per Engels in Anti-Dühring:
If the crises demonstrate the incapacity of the bourgeoisie for managing any longer modern productive forces, the transformation of the great establishments for production and distribution into joint-stock companies and state property shows how unnecessary the bourgeoisie are for that purpose. All the social functions of the capitalist are now performed by salaried employees. The capitalist has no further social function than that of pocketing dividends, tearing off coupons, and gambling on the Stock Exchange, where the different capitalists despoil one another of their capital. At first the capitalist mode of production forces out the workers. Now it forces out the capitalists, and reduces them, just as it reduced the workers, to the ranks of the surplus population, although not immediately into those of the industrial reserve army.
But the transformation, either into joint-stock companies, or into state ownership, does not do away with the capitalistic nature of the productive forces. In the joint-stock companies this is obvious. And the modern state, again, is only the organisation that bourgeois society takes on in order to support the general external conditions of the capitalist mode of production against the encroachments as well of the workers as of individual capitalists. The modern state, no matter what its form, is essentially a capitalist machine, the state of the capitalists, the ideal personification of the total national capital. The more it proceeds to the taking over of productive forces, the more does it actually become the national capitalist, the more citizens does it exploit. The workers remain wage-workers — proletarians. The capitalist relation is not done away with. It is rather brought to a head. But, brought to a head, it topples over. State ownership of the productive forces is not the solution of the conflict, but concealed within it are the technical conditions that form the elements of that solution.
What Engels is saying is that n economy can be completely controlled by the State and still be capitalist. But what makes it capitalist?
The existence of the value form. From Capital, Chapter 26:
In order to be sold as a commodity in the market, labour must at all events exist before it is sold. But, could the labourer give it an independent objective existence, he would sell a commodity and not labour. Apart from these contradictions, a direct exchange of money, i.e., of realized labour, with living labour would either do away with the law of value which only begins to develop itself freely on the basis of capitalist production, or do away with capitalist production itself, which rests directly on wage-labour.
The law of value is the psychological poing of comparison between two goods in the market. As long as this law of value – as long as codified comparison of goods and things – exists, society will be capitalist.