The urbanization of Piazza del Carmine begins in 1330. In 1427 the first records of the property of Messer Michele d’Arrigo di Gardo appeared, with more or less the same dimensions as the present. In 1427 the house was sold to Giacchinetto di Ricciardo Cavalcanti and again in 1472 to Niccolò di Zanobi Borgherini for 1070 gold florins. During that period the house wass transformed into a splendid palace and the tabernacle in Piazza Piattelina was restored.

In the XVI century the building had reached such an importance that it was incorporated into the maps of the city, along with other important monuments of the area such as the churches of Santa Maria del Carmine, San Frediano, la Nunziatina e San Salvatore a Camaldoli. Because of its vicinity to the Pitti Palace, the building became the property of the functionaries of the Medici and Lorena families.

In 1574 the Alidosi-Ciancolini bought and lived in the building, transforming and beautifying it. The garden was one of the loveliest in the Oltrarno area as can be seen from a drawing by Monsignori dated 1584 and named “historic garden”.

In 1632, because of the lack of care of the last descendant of the Alidosi family, the building and its garden were put up for sale in Monte di Pietà and bought three years later by Count Lorenzo di Giò del Mastro, a gallant Medici general

In the register of inhabitants of San Frediano published in 1672 the house was registered as occupied by the family of the Marquis Alemanni and from this date until 1786 there were no more recorded changes of ownership.

In 1786 the property passed as credit to the Cavalier Andrea de’ Rossi di Bergamo, who allowed it to fall into a state of disrepair.

Only in 1790 was it taken over by the Marquis Giuseppe di Lorenzo Ginori, who also bought another two buildings in Piazza Piattellina, enlarging the property and renovating almost completely. It is in this period that the first floor of the building and the grand serene stone staircase are frescoed. The count also enriched the garden with rare and exotic plants, as was the fashion of the time, and installed a gazebo and Kaffeehaus. A high white wall was built separating the garden from the Piazza, over the top of which can be seen the tops of the trees, but through which one can only pass by using the large door. This wall will infuriate the artist Ottone Rosai (1895-1957), who painted it a grand total of eight times.

The Ginori family never lived in the building, preferring to rent it to foreign visitors. In 1814 it was occupied by John Sandfor and in 1827 by the Craufurds, Scottish liberals who will welcome to the house “patriots from all over Italy”. Here, on 11th October 1827 Giorgina, future wife of Aurelio Saffi, was born, fact recorded by a plaque on the front of the building. Amongst the british guests of the house there was also the well known Lady Walpole, Countess of Oxford.

In 1834 the Marquis Leopoldo Carlo Ginori sold the entire property to Prince Giuseppe Rospigliosi, of an important family with origins in the tuscan town of Pistoia, who, later on, for hereditary reasons, will add to his own the second surname of Pallavicini. The double coat of arms is still visible today on the front of the building. Following events document various changes of ownership in which the Rospigliosi no longer take part.

November 1st 1901 saw the arrival in the building, as tenants, the Franciscan Missionaries of Mary. An anonymous association Fribuorgeoise bought the building in 1921 and, ten years later on February 28th 1931, donated it to the Generalate of the Institute in Rome. In 1958, by royal decree, the house became the property of the Franciscan Missionaries of Mary. After the flood of 1966, there were many restorations and important changes made to the inside of the building.

After 400 years, in 1996, there was a total restoration of the facade and the roof of the building and later in 2000 of the serene stone staircase and the frescoes of the late 1700s.