I'm surprised nobody has included "Cool Hand Luke" on their list. Paul Newman was absolutely superb in it.
Also, while not a "true" prison film, I'd list "One Flew Over the Cuckoo's Nest" here.
Yipes, you're right. "Cool Hand Luke", one of moviedoms greatest quotes, "what we have here, is a failure to communicate".
Cuckoo's Nest can certainly qualify as a prison movie. McMurphy was incarcerated for evaluation. "Juicy Fruit". Great film, great cast and performances.