A) First, let's explore the problem in more detail:
The difficulty with both the presented scenarios is that they only decide to perform a test after noticing something about the data -- and then they test the hypothesis that the data led them to on the same data.
When this happens, the test no longer has its nominal properties.
To extend the more "boring" example (and I do it because the probability model is less ambiguous), imagine that ten thousand students each carry our the experiment "roll a six-sided die twenty times".
Further imagine that if a number comes up eight times or more, the students will says "that seems unfair" and so test it.
Let's ask ourselves two questions
(i) If the dice are all fair, what proportion of students will say "that seems unfair"?
(ii) For the students who carry out a test, what's their type I error rate? (i.e. what proportion will reject the null?)
For (i) I just did three sets of simulations of 10000 such students, and got 6.74%, 6.93% and 6.62% getting enough of one face to say "that doesn't seem fair".
For (ii), we have to figure out what test they'll apply and what the nominal significance level they conduct it at might be.
I presume the students are specifically testing for too many or too few of the particular face they noticed a problem with. The sample size is too small to use a normal approximation on a proportions test; if we apply a binomial test, a two tailed test can't be symmetric (equal probability in each tail), but we can get 2.6% in the lower tail and 1.1% in the upper tail by rejecting with 0 or 8+ on the face tested (even though none of the people who test will actually have a zero, since our scenario didn't have students with 0 for a face trying to test it). So the overall nominal significance level is 3.7%.
But what's the actual rejection rate for people who carry out the test (the true type I error rate)? Well, under our rejection rule they all reject ... so it's 100%!
I recall working on a real-world problem like your scenario for someone once; the person's employer conducted random drug testing of drivers, and the particular person had been tested very many times -- he was certain another employee was making trouble for him, leading him to be tested much more often than randomly. If you work out the a priori probability he would be tested that many times, it seemed like something odd was going on. But if you worked out the probability that some employee would be tested that many times, it turned out to be not at all surprising. And of course to that employee it would look strange, even though nothing need be going on. ... and of course the unluckiest of all the (>100) employees would be the one asking what was up.
B) So what can be done? The easiest way is to slightly modify the example. Instead of saying 'that doesn't seem fair, let's test that data', we instead say 'that doesn't seem fair, let's specify exactly what we want to test, and then get some new data to test it with'. Then your examples will work fine!
A store manager assigns shifts to six different employees, including you. After a month on the job, you get the impression that you're getting the "bad" shift way too often. The manager claims that he assigns shifts randomly, namely that each employee has an equal chance of getting assigned the "bad shift." ... so you decide to collect data for the next 20 work days, and you find you're assigned the bad shift eight time. Complete a hypothesis test to assess this claim.
A similar modification can be made to the dice scenario