Are there any math journals or magazines with math problems for middle school or university ?
The various journals listed in the MSE question here can help with the university portion. These include, but are not limited to, the journals:
The American Mathematical Monthly (AMM)
College Mathematics Journal (CMJ)
Math Horizons (I especially like this journal)
The MSE link above contains descriptions of each; I would say that the AMM problems are generally quite difficult, and/but the Math Horizons problems could be fun to try at a variety of math st/ages.
As mentioned in a comment, there are also some NCTM journals. Previously, there were three different journals that covered up through [something like] the second year of college; but, all of those journals have since been combined to form a single publication that launches in January 2020:
I assume that this journal will contain problems, as its predecessors all did; to this end, you may wish to check back issues of NCTM's:
Mathematics Teaching in Middle School (MTMS)
The Mathematics Teacher (MT)
Each of these has, at the least, a set of calendar problems. You can also find a good number of articles around problems; for example, I contributed a selection of problems in an article for MT about roots of polynomials (link without paywall).
Finally, although it is not a journal or magazine, if you are simply interested in problems, then there are lots of resources online. One possibility - since your original question mentions middle school - is to look at past competitions from MathCounts; you can find their problems archived here. At the other end of the spectrum, you can find very challenging problems in the archive of Putnam Problems maintained here. Both of these linked sites also contain sample solutions.