If you test them (and without any ability to reach the Internet, no phones, computers) than it will be an action that makes it hard to over-rely on WA. If you do graded homework, it is the opposite. Thus, I would suggest more tests, quizzes (in class) and less graded homework.
P.s. I think for a very strong student (or a working professional) there is nothing wrong with using WA or even just a handbook. After you are done with a course. But there is a disadvantage in being someone who only can use WA and can not do anything himself. I would include Wolfram, the actual man, and many others in this group. They learned it first analytically and it informs their use of WA, their development of it, their fluency reading physics or engineering derivations, and their analysis of problems in the real world.