Fishing while selecting for big specimens and throwing smaller specimens back leads to the unfortunate result of us creating selective pressure for small fish. Those big fish have, when they are smaller, inefficient reproductive systems which means they have a harder time creating enough offspring in the wild. Even if what you are saying had anything to do with how things actually work (it doesn't), this selective pressure contributes to our favourite fish dying out.