Andrew Bogle was a Black slave on the Hope Estate in St Andrew, Jamaica. He was born about 1801, but nothing is ever said about his parents. He was taken to England in the mid-1820s by Edward Tichborne, who had for some time been managing the estate. Tichborne apparently took Bogle with him without informing anyone of his intentions, which caused considerable annoyance, and employed him in England as his valet. At first Bogle lived with Tichborne and his new wife at Upton House, near Poole, in Dorset. Influenced by his employers he was baptized in the Roman Catholic Church; the Tichbornes were one of the oldest Catholic families in England. At Upton, in the 1830s, he married Elizabeth Young, a White English woman, who had been a nurse in the household of Tichborne's aunt; the marriage took place in the Anglican church in the neighbouring village of Great Canford, as at that time Roman Catholic marriages were still not legally valid. Bogle had two sons by his first marriage, John and Andrew, but his wife died in 1845.
Andrew Bogle continued as Edward Tichborne’s valet when in the mid-1840s Tichborne inherited the Tichborne baronetcy and property. The family moved to Tichborne House in Hampshire, and Bogle’s two sons, John and Andrew were sent to a Catholic school near Reading. In 1853 Sir Edward died, and, shortly after, Bogle was able to retire with a small pension. He decided to emigrate to Australia, possibly encouraged by another retired Tichborne servant who was living in Sydney. Before leaving he married, at St Peter's Roman Catholic Church in Woolwich, Jane Fisher, the Dorset-born daughter of a soldier, who had been a teacher at the village school near Tichborne House. His son Andrew went with them; John was apprenticed to a chemist in Nottingham, but also travelled to Australia when his bad conduct brought about the end of that arrangement.
In Sydney, Bogle for a time kept a restaurant, and his sons started on careers; John continued as a chemist and Andrew junior became a barber and hairdresser. A third son, Henry George, was born in 1856. However, in 1858, a common family tragedy befell Bogle. His wife, then aged 43, gave birth to another son, Edward, but died three weeks later of 'uterine hemorrhage'; the baby only lived for two months. Bogle himself began to suffer severely from rheumatism.