The club opened in 1947 in Nelson House. It was founded to provide a first-class meeting place for the servicemen and women of the Second World War. When it opened it was the only club in Cheltenham which admitted women members.
In 2000 it was decided that the Nelson House building was too expensive to run. The Victory club was demolished in February 2004 to make way for new townhouses. Nine luxury homes have now been built on the site. The Victory club moved to new premises in Burlington House, Lypiatt Road in 2002.