I have used (and still use) a Huion H420 graphic tablet (it costs around 40$). It has pressure recognition which really helps the software understand and interpret your writing and give a nice output. (Some software does not take pressure recognition into acount and you don't get the full benefit of your tablet in that case).
After the first session (say one hour), you should both be pretty much at ease with the way writing works with that tablet. It's like normal writing, except you have to do it in a constrained space, but in my experience, it quickly becomes second-nature so I wouldn't worry about that.
The software I liked the most was Microsoft OneNote 2010 (but it also works in Microsoft Word 2010 and I assume more recent versions). I think you can have access to the free online version of these softwares by making a Microsoft account, if you want to try it out.
These software recognized the handwritting pretty well. The curves are not converted into rough lines, even if you write really fast, so you don't have to spend 60 seconds writing one sentence or one equation in order for it to be readable.
Then it's a matter of finding software to "share-screen". I have used Skype in the past but a downside is that only one screen could be shared at a time (I do not know if this is still the case and if this has been improved). So for example, I could share my screen with someone else, give explanations, but they couldn't show me their work or write on top of what I wrote without me ending my screen sharing and them opening theirs (so it can get tedious if you need to switch often).
Things to consider are the synch. Ideally, audio is in synch with video (so that your nephew hears the explanations and sees them written out simultaneously) and your tablet is in synch with your video (for obvious reasons). If your sessions are to last an hour or more, then even a 2 second delay might become annoying. One thing that's happened to me in the past is that I always cut the other person as they were asking questions; all of this due to bad sync (although that was not with Skype). Turning off webcam could help in this situation.
Last I checked Skype is also free software so you could try that out today with him and see how your connection is and get a feel for the screen sharing. For example, try to draw things in Paint and ask your nephew if things appear smoothly enough for his taste on his end (does the image refresh fast enough or is it too sloppy).
I believe there are some "online whiteboards" that are free can be shared in real-time, but all the ones I found where nowhere near as precise as OneNote was. For example, writing at a fast pace would be perfectly readable in OneNote but complete gibberish using other software. (A $4$ might look like a $9$ because every curve you draw is turned into a rough "line".) I believe there might be two possible causes to this:
- The software isn't made to recognize as much precision as my graphic tablet is capable of (and it's a really cheap one)
- There are issues with the way data is transferred due to poor Internet connection
As for the "relational aspect", sure it's different than having him work next to you, but there is no "stranger barrier"; you already know each other and have worked together in the past and if that worked out, I don't believe that sessions being online will make things that different. You say he liked working with you, so it's not as though you had to discipline him into not going to Facebook while you both work. It's definitely do-able.