Quite simply, it depends on who you ask. There are two competing definitions. One comes from STATS, Inc., the other from SportsTicker.
STATS, Inc Definition:
The hold was invented in 1986 by John Dewan (now an executive at STATS, Inc.) and Mike O'Donnell, who worked together on The Chicago Baseball Report.
A Hold is credited any time a relief pitcher enters a game in a Save Situation, records at least one out, and leaves the game never having relinquished the lead.
This definition comes from the STATS Baseball Scoreboard: 1996. Notice that a pitcher cannot get a hold unless he leaves the game before the game is over. You cannot finish a game and also get credit for a hold. You'd get credit for a save instead.
Further, a pitcher can get a hold if he pitches three innings or more, regardless of how big his team's lead, as long as the lead is intact when leaves the game.
Example: Stinky Anderson enters the game with his team ahead 12-6 after 5 innings. Anderson pitches the 6th, 7th and 8th innings. Anderson gets credit for a hold so long as he didn't relinquish his team's lead, and his team does not relinquish the lead in the 9th inning. The score could be 12-11 at this point and Anderson would still get credit for a hold.