When the Vatican announced that it would beatify Pope Pius IX during its Jubilee celebrations last year, the Brown University historian David I. Kertzer had reservations. He knew that Pius IX, a staunch 19th-century critic of modernity who led the Roman Catholic Church during a tumultuous period that included the demise of the Papal States and the unification of Italy, was a favorite of the church's conservative wing.

But Mr. Kertzer, an expert in 19th-century Italian history, also knew that when it came to Jews, Pius IX's behavior had sometimes been less than saintly. Interviewed on Italian national radio a few days before the beatification ceremony, Mr. Kertzer cited remarks Pius IX had made to an audience of Catholic women in 1871, in which he referred to Jews as ''dogs'' who went around ''barking in all the streets'' and ''molesting people everywhere.''

At a news conference the next day, a Vatican spokesman dismissed Mr. Kertzer's evidence (from a book of the Pius IX's speeches), saying no ''serious historian'' had ever mentioned these remarks.

To Mr. Kertzer's ears, this reaction was in keeping with other official church pronouncements on anti-Semitism in the past: with rare exceptions, it simply didn't exist. In his polemical new book, ''The Popes Against Jews, the Vatican's Role in the Rise of Modern Anti-Semitism'' (Knopf), Mr. Kertzer takes the Vatican aggressively to task, arguing that it has yet to acknowledge a history of church-sponsored anti-Semitism that helped pave the way for the Holocaust.

The book won't be in stores until Sept. 18, but battlelines are already being drawn. Some experts dismiss Mr. Kertzer's argument as a tendentious misreading of the facts. Others say he is merely bringing the ugly truth to light. But one thing is sure: in advancing his claim, he steps into the middle of a debate that has been gaining momentum for 40 years and lately has reached a flash point. From the recent skirmishes over Pope Pius XII -- who headed the church during World War II and has been portrayed as both a saver of Jewish lives and a heartless anti-Semite -- to conflict over how to interpret a 1998 church statement on the Holocaust, the history of the Vatican's attitude toward Jews has never been more in dispute.

In the latest sign of tension, a two-year-old panel of Catholic and Jewish scholars, jointly appointed by the Vatican and an international Jewish organization to review papal records relating to World War II, disbanded last month, its work unfinished. (A news release issued by the panel's Jewish coordinator blamed the Vatican, saying its refusal to grant access to relevant archives effectively hamstrung the scholars.)

But while much of the recent scholarly attention has focused on Pope Pius XII and the period around World War II, Mr. Kertzer goes much further back, taking on the Vatican's entire modern history. Drawing on material from newly opened archives of the Roman Inquisition (which began in the mid-1500's and didn't peter out until around 1900), he constructs a picture of church-sanctioned prejudice and oppression ranging from forced baptisms and conversions in the 19th century to expressions of virulent racial hatred in the 20th.

''If you're interested in what role the church played in making the Holocaust possible,'' Mr. Kertzer said in a telephone interview, ''it's decades and decades of demonization of Jews.''

In the book's introduction, he attacks the Vatican's 1998 statement on the Holocaust, arguing that the distinction it draws between the church's historic anti-Judaism, defined as ''longstanding sentiments of mistrust and hostility'' and Nazi anti-Semitism, ''based on theories contrary to the constant teaching of the church on the unity of the human race'' is not supported by the facts. He writes, ''If the Vatican never approved the extermination of the Jews -- indeed, the Vatican opposed it (albeit quietly), the teachings and actions of the church, including those of the popes themselves, helped make it possible.''

Reached by telephone, a Vatican spokesman declined to comment on the book, saying that he had not read it. But scholars who have say its publication is a significant event. ''It's an important book,'' said Michael Marrus, a historian at the University of Toronto who was one of three Jewish scholars on the Vatican panel that disbanded last month. ''Unlike a lot of writing on the subject, Kertzer knows what he's talking about. He's seen stuff nobody else has. Its strength is showing the power of anti-Jewish opinion even at the center of the Catholic Church.''

James Carroll, a former Catholic priest and the author of ''Constantine's Sword'' (Houghton Mifflin), a critical history of the church's treatment of Jews, agreed. ''The Vatican is obviously trying to backpedal as fast as it can away from the dark history of the Catholic Church,'' he said. ''Kertzer is telling the truth.''

But Eugene J. Fisher, the associate director of the Secretariat for Ecumenical and Interreligious Affairs at the National Conference of Catholic Bishops, questioned Mr. Kertzer's conclusions. ''There is a distinction between the church's anti-Judaism, even at its worst, and Nazi anti-Semitism, which led to the death camps,'' he said. ''If the teachings of the church had flowed to genocide, that would have happened around 1300 or 1400 when the church had real political power.''

Mr. Kertzer begins his account in 1814, with the restoration of papal rule after the routing of Napoleon's army. While elsewhere in Europe Jews were increasingly free to live as they wanted, Jews in the Papal States were locked into cramped ghettos at night, forbidden to practice law or medicine, hold public office or hire Christian servants. Some were forced to undergo baptisms and conversions as well.

If a Jewish child was known to have been secretly baptized, Mr. Kertzer says, he or she would be taken into police custody, given a new name and raised a Catholic. (One such case from 1858 involving a 6-year-old boy formed the basis of Mr. Kertzer's previous book, ''The Kidnapping of Edgardo Mortara,'' which was a finalist for the National Book Award in 1997.)

These practices, Mr. Kertzer argues, were the inspiration for the racial laws enacted by the Nazis and the Italian Fascists in the 1930's. After the fall of the Papal States in 1870, he writes, the church's hostility toward Jews began to take another, in some ways more disturbing form: no longer simply loathed as unbelievers, Jews, now freed from papal rule, became hated symbols of secular modernity.

As proof, he cites Catholic publications with close ties to the Vatican, including ''L'Osservatore Romano,'' the Vatican's daily newspaper, and ''Civiltà Cattolica,'' the Jesuit biweekly considered to be the unofficial voice of the Pope. Among the charges leveled against them, Jews were accused of being world dominators, tyrants, thieves, liars, communist conspirators and money grubbers. They were also said to engage in ritual murder or blood libel, which involved draining the blood of Christians for use in Passover bread.

By the turn of the century, some Catholic reporters were using the term ''anti-Semitism'' with approbation. ''In its original form, anti-Semitism is nothing but the absolutely necessary and natural reaction to the Jews' arrogance,'' the Vienna correspondent for ''Civiltà Cattolica,'' wrote in 1922, adding, ''Catholic anti-Semitism -- while never going beyond the limits of moral law -- adopts all necessary means to emancipate the Christian people from the abuse they suffer from their sworn enemy.''

Such tactics, Mr. Kertzer insists, were condoned by church officials at the highest level. Drawing on correspondence from the period in the Vatican archives, he describes how the Holy See gave behind-the-scenes support to the overtly anti-Semitic Austrian Christian Social party, bestowed a papal blessing on the author of an anti-Semitic book and, in 1900, turned down a request from the Archbishop of Westminster and several prominent English Catholics to issue a public refutation of the Jewish ritual-murder myth.

Mr. Marrus, whose review of Mr. Kertzer's book appears in the September/October issue of The New Leader, called some of these findings ''shocking,'' saying that the book shows ''how deeply involved the leading organs of public opinion and the popes themselves were on questions like ritual murder.''

Other scholars disagreed. ''Kertzer's taken the worst examples said and done in the name of the church and argued that they were central to the teachings of the Catholic Church,'' said Ronald Rychlak, the associate dean of the University of Mississippi law school and the author of ''Hitler, the War and the Pope'' (Genesis Press), a sympathetic portrait of Pope Pius XII. ''There were anti-Semitic articles and editorials, but he discounts popes' distancing themselves from that.''

Indeed, where others have portrayed his predecessor, Pope Pius XI, as a courageous defender of Jews -- one who tearfully told an audience of Belgian pilgrims in 1938: ''Anti-Semitism is inadmissible. We are all spiritually Semites'' -- Mr. Kertzer depicts him as a pontiff whose moral outrage was tempered by his allegiance to traditional church culture, where villification of Jews was routine.

In Mr. Kertzer's view, the famous hidden encyclical against anti-Semitism commissioned by Pius XI shortly before his death in 1939 included anti-Jewish stereotypes and was ''less than a ringing condemnation.'' (The encyclical was never published: Pius XI died without releasing it, and his successor, Pope Pius XII, who maintained diplomatic ties with Hitler, did not pursue it.)

But Mr. Fisher disputed this assessment, saying: ''There's no evidence that Pius XI ever saw the draft of the encyclical. You can't blame him for what's in it. He started a process, it reached his desk and then he died. This was a pope who was getting angrier and angrier at the Nazis and more and more willing to speak out.''

In the end, Mr. Kertzer's book seems more likely to incite new controversies than to resolve old ones. But that, said Rabbi A. James Rudin, the senior interreligious adviser for the American Jewish Committee, is not necessarily a bad thing.

''This has been a period of revolutionary change in Catholic-Jewish relations,'' he said. ''Instead of isolation and suspicion, you now have passionate engagement. And the last major issue that has to be resolved is the whole record of the Catholic Church during World War II and the period leading up to it. There are going to be more books, more questions, more pressing for documentation. And that is the way it should be.''