At the time of Robert's birth, Rosanna had been sold back to Lady Douglas, who had owned her previously; James Wedderburn's brother, Sir John, had protected Rosanna against his brother's violent behaviour, and Lady Douglas, who was Robert's godmother, always protected him. Part of the agreement of the sale of Rosanna to Lady Douglas was that the child she was carrying should be free from birth, so Robert was born free.
The whole story of the Wedderburn family's relationships in Jamaica at this time seems fairly complex. James Wedderburn appears to have been far more vicious in his relationships with slaves, and Blacks and Coloureds in general, than other members of the family. His later rejection of Robert Wedderburn as his son seems to indicate, however, some particular animus against Robert. Robert himself mentions that his eldest half-brother, John, son of James Wedderburn's Scottish wife, who came to Jamaica, presumably in the early or mid-1790s (and died here in 1799) 'acknowledged his father's tawny children, and, amongst them, my brothers as his brothers. He once invited them all to a dinner, and behaved very free and familiar to them.' This was long after Robert left Jamaica.There seems to be much more to the Wedderburns in 18th century Jamaica than the disgusting behaviour of James Wedderburn.
Robert Wedderburn grew up in Kingston under the care of his grandmother, known as 'Talkee/Talky Amy' who was a slave who traded goods for her master, and for other merchants on commission. She was so much in the merchants' confidence that she traded smuggled goods for them. At the age of 11 he saw his 70 year old grandmother flogged within an inch of her life, accused of witchcraft (or presumably obeah).
Sometime in the 1770s he left Jamaica and served as a sailor on British ships, as did many other Black and Coloured men at the time. He took on board the grievances of seamen of the time, especially against the brutal floggings that were customary, and added the sailor's lingo to his boyhood's 'Jamaica talk'.