This depends on what information you are asuming that the attacker has.
First, let's asume that the attacker is blind, and perhaps trying to crack a large dump of breached accounts, without knowing that you used that specific algorithm. Then you would be better protected if you discarded
123456 if it comes up, or more realistically, passwords that are only lowercase, dictionary words, etc. This is for the simple reason that any attacker is bound to try those first since in general they are more common.
On the other hand, if the attacker knows that your algorithm was used, discarding passwords will only make her job easier. If you dump 10% of the passwords, that is 10% she does not have to try, saving her 10% of the time on average.
So which one of these two scenarios are more likely? I would say the first one in most cases, but only you can determine what your threat model is.
A mathematical example
Let's do some math to see how many passwords you would actually drop. Let's say that characters are picked from three groups (upper case, lower case, numbers + special characters) and let's say that there are an equal number of character in each group. Furthermore, let's say that you require passwords to have at least one character from each group. What fraction of passwords would you drop?
If I get my math right, it is
L is the length of the password. For
L=10 you get 5%. For
L=20 you get 0.1%.
Unless you are completely sure that the only threat is an attacker that knows how the password was generated (I find it hard to imagine how you could be that), I would say that reducing the search space by 0.1% is worth it.