Not unlinking the readline *will* cause problems, this is a rather frequently used lib. You should do this.
But I also think I now know the actual problem: The readline library (not Term::Readline::Gnu) is not found when you try to start polymake. The fact that linking from /usr/local/lib to the libreadline from homebrew (which is what the forced link command does) suggest that it is actually installed, but not found during runtime.
Can you do the "otool -L" command on the page that you linked? This will list all libraries loaded by Gnu.bundle during runtime. One of them should be libreadline.dylib (maybe additionally with a version number in the name). It should come with a full path. Can you check whether the library exists at the specified path? I think it should point somewhere into /usr/local/opt, but currently I can't access my MacOS installation with homebrew.
Andreas