Jim was born on December 31, 1933, in Norman, Oklahoma, one of three children born to Hardin Ballard and Evelyn Glasco Ballard. Jim lived most of his life in Purcell, attending Purcell Schools and graduating in 1951 from Purcell High School. In 1953 Jim married Anita Abrams and together they raised two sons, David and Stephen. Jim was active in sports playing football, basketball and baseball. He graduated from the University of Oklahoma in 1958 with a degree in Geology. In 1962, he received his LLB from the University of Oklahoma and joined his father Hardin Ballard in the practice of law in Purcell. In 1963, Jim and his father Hardin Ballard were successful in appealing a federal court decision on reapportionment of the Oklahoma legislature. He also served as Assistant District Attorney in McClain County during the 1970’s. While attending college, Jim rough-necked and spent two years working for Baroid Well Logging Service. He loved all kinds of sports and coached peewee football and baseball for many years. He was instrumental in helping build the peewee baseball field during the 1960’s and was president of the Mary Simpson PTA in the early 1960’s. In 1962, Jim was at the forefront of a group to install grass greens and enlarge the Purcell Golf Course and was champion of the first annual Purcell Golf Association tournament in 1969; was president of the Purcell Rotary Club in 1964; ran unsuccessfully for the State Senate District 22 in 1964; and helped organize the Mid-America Vocational Technical Center at Wayne in 1968.