What would you put in a Computer Science Curriculum?

by Dan Zambonini

We get a number of resumés trickling through every week, with a fair proportion coming from Computer Science graduates.

I look at the list of modules they've studied, and although they sound very interesting, there seems to be little relevance to the current jobs market:
neural networks, computer vision, artificial intelligence, robotics, compiler engineering, machine learning, quantum computing,
Bayesian networks, embedded systems
... How many of these graduates will go on to regularly use this information? And these aren't dodgy
establishments teaching these modules; I took this list of modules from the curricula of the 'Top 5' computer science departments in the UK
(graded by a respectable national newspaper).

Part of this mismatch (between education and skills) could be caused by the misguided notion that 'Computer Science' graduates don't
necessarily go into 'Software Engineering', so are taught a more scientific approach, rather than vocational. However, the
Computer Science graduates I know have software/programming careers, not science. Interestingly, from a random selection of
about 10 or 15 university courses I researched for this weblog entry, only 2 offered Software Engineering.

The 'science' bias could also account for the large number of mandatory maths modules in these courses; one of the top 5 universities
has 25% of its mandatory modules dedicated to mathematics (including 1 mainly on 3D maths). Another top university states that the
final year project is to 'Develop a game'. Great for attracting students, maybe not a great idea for getting them a job? I just
did a search on monster.com; there were 31 'games programmer' vacancies, and more than 1000 for 'software developer', and more than 1000
for 'web programmer'. Not much use knowing how to handle three dimensional diffuse lighting vectors when
you're trying to create a customer feedback form for an insurance company.

And, quite surprisingly, only 1 course mentioned XML, which was in an optional module. Is there any modern software
these days that doesn't use XML? So why can't computer science graduates tell me when to use SAX and when to use DOM?

So, what would you include in a modern Computer Science/Software Engineering course? I've started jotting some draft notes
below (in no particular order, this is just a stream of conciousness...)

  • The basics of Programming (variables, data types, references, pointers, scope, error handling, iteration, core algorithms - searching, sorting, etc.)

  • Basic mathematics, basic statistics

  • Patterns and Anti-Patterns (With real world examples, not just theory)

  • Real world Databases (Normalisation and De-normalisation, SQL, Indexing)

  • Basics of good code architecture: Loose Coupling, etc.

  • OO Design, Interfaces, etc.

  • The importance and tools of Planning: Spec'ing,, UML etc.

  • Architectures: client/server, SOA, P2P, etc.

  • A 'Big' language or two (Java, C#, C/C++)

  • A scripting/'agile' language or two (PHP, Perl, Python, Ruby)

  • XML (DOM/SAX, XSLT/XPath, etc.)

  • Economics, Business Studies, Costing Projects, Commercial pressures

  • Copyright, Privacy, Data Protection

  • Project/Time Management

  • Internationalisation, Localisation, Encoding, Unicode

  • Grammar, punctuation, concise and clear writing

  • Interface Design, Usability, Accessibility, HCI

  • Security

  • Code Reading

  • Common Protocols (TCP/IP, HTTP, SMTP, FTP)

  • Testing, Debugging, Performance, Re-factoring

  • Problem analysis

  • Source control, change management

  • The typical Software lifecycle

  • Metadata, Information Architecture, etc.

  • The basics of GIS

  • Touch typing

  • Health and safety (nutrition?)


2005-09-08 05:57:15
You might take a look at Informatics, such as taught at my school ( http://informatics.indiana.edu ). While Informatics is less focused specifically on programming, one "cognate" (area of specialization) available to majors is in Computer Science, which, combined with the required Informatics curriculum, covers a lot of the things you list.
2005-09-08 09:12:18
Programming, not computer science
As a colleague points out, they're teaching computer science, not programming.

I'm amazed at the bad practices that students seem to have no problem with. The concept of DRY (Don't Repeat Yourself) is one that seems to elude many. I got some sample code yesterday from a student, fresh from graduation, with a 10-case switch statement with the same five-line block of code and one differing line in each.

The other one that amazes me, and that you mention in your list, is version control. There's no reason why every programmer in the world shouldn't be able to use Subversion.

2005-09-08 09:54:02
A good development environment
The same question occurred to me while writing the upcoming O'Reilly book "Practical Development Environments" (http://www.oreilly.com/catalog/practicalde). I didn't find anywhere that taught students what to look for in their software tools.

In addition to source control, I'd also suggest:

- build tools (make, Ant, ...)
- test environments (harnesses)
- bug tracking tools
- documentation environments
- managing releases

All pretty prosaic, but very irritating if they don't work well.


2005-09-08 10:25:52
Courses For Consideration
1. Technical writing and research skills course - I value reading and writing so much more now.

2. Legal, ethics, and politics course - You need to be informed of the current issues.

3. Community service (seminar course) for a semester.

2005-09-08 11:29:21
Nothing wrong with CS
I wouldn't change a thing in the Computer Science curriculum, if anything I would add more classes on AI, compiler theory, and so forth. For one, if a student can make it through these classes, then they should also be able to figure out when to SAX as opposed to DOM with the greatest of ease. After all, college is not a trade school, it should be teaching students how to think and learn, and not just a few skills that will soon be outdated. If you want skilled labor, look at students graduating from vocational schools. Also, Computer Science has very little to do with programming (At my school only two mandatory classes were programming related). If you want to be a programmer, do a Software Engineering degree. If you want to be a scientist however, do CS. I'm in the latter group, I love CS, I love AI theory, and I love topics such as bio-inspired computing. I chose to do a CS degree and to become a scientist. My friends who wanted to program chose to do a degree in MIS, and everything seemed to work out fine with all of us.
2005-09-08 15:35:12
Lack of "machine" and also more underlying science
Very quickly, two comments:

1. Over the years I've noticed the near complete demise of teaching of how computers works at near-hardware level. CS students were once taught assembler (and even machine langauge) and basic hardware logic. Today I'd add more concepts like caching, pipelining, OOE, etc. The point is that do todays students really have any notion of what the computer is actually doing? This impacts on a lot of performance-related issues. For example, its hard to appreciate how a good compiler contributes to code performance unless you have some appreciation of what the hardware is up to. (You could have a similar thread on OS structures now that I think about it.) Some might argue that this isn't needed much anymore, but in many areas its still vital.

2. I've always thought that universities should emphasis the underlying theory/structure, with the practical as just a necessity, not the essence, unless its the "lower stream" course for people in other disciplines to pick up programming. For a CS course, ompiler theory, OS structures and approaches, number representation (incl. IEEE), parsing theory, etc., etc. And, err, you've listed nothing on algorithms??! better not start me on that ;-)

Your course so far looks like a "lower stream" practical course to me, with the CS students to take additional course(s) that teach the actual computer science, as opposed to programming (as croach touched on).

myc18's suggestion of technical writing is excellent, but I'd prefer to see that taught as a short course by the appropriate Arts dept. Ditto for the legal side of things, but done by the Law dept. of course.

andy, I agree with the DRY and coding practice issue. I've seen published graduates from the "new" bioinformatics crowd with code that horrifies me.

2005-09-08 15:59:02
Lack of "machine" and also more underlying science
Completely agree with the low-level stuff, I'd forgotten about that. I really like this Joel Spolsky article that talks about why you should be aware of what's going on underneath: http://www.joelonsoftware.com/articles/fog0000000319.html

To be fair, I did mention searching and sorting algorithms, but I did sort of gloss over them...

2005-09-08 21:31:47
Lack of "machine" and also more underlying science
Sorry I missed your ref. to core algorithms--reading in a hurry.

Thanks for the link to the article, although I'm so busy it might be a week or two before I get a chance read. Joel usually writes well. Lightly skimming it seems to be about strings et al. I've seen some very clever implementation of strings in "C-land" (I've even a fancy one of my own which I've never polished the rough edges off) with some nice coding concepts that'd do well in a CS course.

Simple things like how hardware has evolved and the concepts brought in are useful. Some very basic concepts have application elsewhere and can be surprisingly "deep", e.g. caching, indirect references (in the very early days memory refs. were direct...), various types of look-up tables, etc.

On the subject of history, one of the reasons people like Joel "see" well is that they saw the history pass and understand why things are the way the are. The current schemes are really designed so much as evolved if you think about. I think you get a lot deeper understanding of a subject if you understand its history. The same is true of most fields (I'm a computational biologist and I can say the same is true of biology and most of the "hard" sciences).

2005-09-11 09:01:12
i attend one of those "top 5" universities. you are very much mistaken as to course content. you are also quite mistaken about the point of a degree. a degree is meant to stretch your mind so you can easily adapt to anything you are faced with in the future.

do you really think that xml will be around in 20 years time? i really hope not.

your idea of a cs degree is also bent towards web development. this is hardly the most intellectually challenging of jobs; compared to, say, supporting a stock exchange, or designing an embedded system -- oh, you didn't know these are Real World Jobs? pity.

you fail to see the applicability of ai, compilers, embedded systems, etc? you clearly have a very simple problem domain. perhaps you consider learning how to prove asymptotic bounds to also be a waste of time because everyone just uses the built-in libraries? ever heard of data mining and the semantic web? i gather these might be quite important at some point.

on your list, my degree does not cover: design patterns (these are hardly difficult to learn yourself..) XML, economics (what are you smoking?) i18n. that's really it. everything else is already covered, and we get to do more interesting stuff like ai.

2005-09-11 09:06:19
The problem is in the basics
The biggest need today (be it CS or software engineering) is to teach people how to read a program so that they can fix, utilize, or otherwise work with an existing piece of software.

I think that we need to start off teaching people how to understand programs before they learn how to write them. I would start off with an "Introduction to Programs" that presents basics of program execution and basic structural constructs but NO programming. I would then have some form of "Introduction to Programming" which would build upon this base to teach students how to use what they learned in the previous course to constuct a program.

2005-09-11 09:09:37
Your degree also obviously fails to cover teamwork and interpersonal communications. (i.e. how to convey your ideas and opinions without being a pompous a$$.
2005-09-11 09:18:54
no, it does not. you work in teams from the start.

initially working in pairs on assessed work (such as building a computer from the ground up: literally designing and building ttl circuits, to writing the control/os software.)

later in larger teams (6-7) in higher pressure team competitions constructed in collaboration with industry.

2005-09-11 09:55:15
My CPS Curriculum
I am finishing up my MS in CPS at the moment at a state college in Michigan. At the same time, I don't feel like I know enough to get a real job. I feel like I haven't learned anything.

I'll give you a run down of the classes I took.

Intro to Programming w/ theory
Data structures
Intro to Algorithms
File Structures (mostly COBOL programming)
Windows Programming (or VC++ 6)
Databases (theory only)
Artificial Intelligence (my area of interest)
Computer Architecture (AND/OR gates, basically)
Programming Language Concepts (automata theory)
Intro to OS (learned NOTHING, all theory)

More Algorithms (mostly theory)
Computer Networks (UNIX based)
Graphics (all theory)
Digital Image Processing (basically computer vision)
Compilers (actually built one)
OS (forks, processes, threads, etc on UNIX)
Intermediate AI (more of the basics)
Advanced AI (focused on neural networks, still learned nothing)
Pattern Recognition (data mining, no programming)
Cryptology (all theory)
Biocomputing (all theory)

With so much theory and little programming, is anyone surprised that I feel like I can't program well enough to get a job?

To remedy this, I'm staying in to get another major and five minors:

Cognitive Science Major
Math Minor
Philosophy Minor
Psychology Minor
Business Minor
Communication Minor

I'm focusing on AI, but am learning new programming languages and techniques on my own time. And I'm going to see about internships to get practical experience.

I'm still not sure if I'll be ready, but I'll be much more prepared than I am now.

2005-09-11 10:27:22
The science versus the program
Dan, you’re taking on an important and difficult topic. It’s good to note that just because CS cirricula are manifested in their current form, it doesn’t mean that this is “the natural order” that should never be disturbed.

Perhaps I should state my biases first: I have a BS in Computer Science and an MS in Mathematics. Thus, I’m likely (rightly or wrongly) to be in favor of “theoretical subjects”. One assumption I’ll be making is that (budding) software developers are generally intelligent people, so when I say something such as “concept X is easy to understand”, the reader should take it to be a statement about the abilities of individuals in the above-mentioned group, and not a statement about my own oh so clever self.

I’ll first consider the points with which I agree, followed by my opinions on those points that I disagree with.

The writing skills of most software developers are atrocious (myself generally included). There is certainly a necessity for a course that teaches good writing skills and for one or two courses in the Commercial Sciences.

Furthermore some abstract but important software development concepts are generally ignored in CS cirricula, such as basics of software architecture (i.e. loose coupling, OO Design etc.). However, is it really necessary to spend much time learning about particular architectures, such as P2P? These concepts are quite simple and take little time to grasp.

Although some skills such as debugging, performance testing etc. are very concrete, they are quite essential for any CS student, since most of us ultimately want to translate our ideas into computer programs. Thus, it would be good to have a continuous (i.e. over 2-4 years) low load subject dedicated to these issues.

Some skills such as planning, UML etc. are a good idea, but perhaps students should be expected to learn these skills in a (industry recognized) post-graduate diploma course, since they are quite concrete. Project management, time management and programming methodologies should also be left to specialized diplomas, since the theory of these fields change relatively frequently.

There are a couple of skills that I feel are best left to experts in other fields–we should be aware of the issues pertaining to their fields, but we should realize that these fields are specialized and beyond the scope of CS. I consider UI design and security to fall into this category. I’ve seen people scoffing at UI design, insinuating that anyone can do it, but the sheer amount of bad UI design that is prevalent in Windows and Unix (except MacOS X) worlds tells another story. Security should be designed into a system from the start and it will always be difficult for developers to produce secure code for insecure systems.

My single greatest gripe is that some of the skills you list are either relatively minor (such as touch typing; I was a full touch typist after my 3rd year in my CS degree), or relatively easy to learn.

For example, the details of TCP or HTTP are not hard if the principles of layered protocols are understood (which itself is hardly rocket science, especially since layered abstractions permeate CS theory); should it really be expected of a student to learn the details of the protocols, considering how fleeting such knowledge is with the synaptic attentuation of our overstimulated brains? My opinion is that It’s quickly acquired if needed. I believe the same about database technology, XML, Internationalisation etc.

I’ve always had a gripe with the hype surrounding languages. A good software developer should be able to acquire knowledge of a new language with little effort. Perhaps it would be better to expose students to all the programming models–i.e. OO, functional, logic etc. programming. I think the value of hiring an experienced, say, Java programmer has less to do with his knowledge of Java (which is a pretty simple language), than with his knowledge of the truly massive library and the “ways of doing” in Java. With a system such as .Net, it will probably become very apparent that the langauge matters less than managers think, since .Net libraries can be accessed from any .Net langauge.

Finally, as a little (flame inspiring) nitpick that probably won’t contribute at all to my other opinions, I don’t want law mixing with CS. Nothing illustrates this better than the EU legislation regarding the patentability of software methods. Many lawyers imagine this to be a good thing, which is completely at odds with the way most software developers see it. The same is true of many other so-called “IP” laws.

As my closing statement I’ll say that many of the skills you list are undoubtably important in the working place. However, I don’t consider most of them to be fun or interesting, and certainly not mind expanding as some most “theoretical” CS subjects can be. I suspect many others feel as I do. Software development should be fun. I believe that the tendency to substitute “development” in “software development” for “engineering” is a sign that the fun is being squeezed out and that overbearing processes and “real world” concepts are making their way into our lives.
My desire is that we should always endaevour to remove as much “process” and as many “commercial concerns” as possible without forgetting that we work in environments which are business orientated. Sure, the business people are the ones with control over our paychecks, but no good software can arise in an enviroment laced with such threats and with distractions such as useless meetings, senseless project management spreadsheets that must be kept up to date etc.

So, I would like to see CS degrees remain theoretical and (what I consider to be) mind expanding.

2005-09-11 11:04:01
We're hurting for real Software Engineers
I really have to disagree with Mr. Zambonini. One of the courses listed as not useful for getting a job was "embedded systems". Ahem. In case some folks haven't bothered to notice, something like 70% of all the "computers" on this planet are invisible. Now consider how many computers are "visible", and that works out to be a BIG number. Seems to me that there might be a bit of disconnect here. But, all that aside, I manage a lab developing software for space missions, and before that I managed projects developing and testing software for things like engine control systems on commercial aircraft. When I posted a job opening I would have been thrilled to see a resume with "embedded systems" in the list of courses taken by the applicant. I'd be thrilled to see one now. Every time I post a job opening at the lab I get something like 40 resumes. Of those I would say that 90% list things like web server maintenance, Java and web page design as major skills (with a smattering of Visual Basic and C++). Besides the fact that I can't figure why these people even bothered to apply (the job posting clearly states that we're looking for things like embedded and real-time knowledge, real-time image data processing, ANSI C programming, and robotics), the ones I do end up picking to interview can usually do a perfect "deer in the headlights" impersonation when I ask them about things like assembly language, real-time deadlines, coordinate transform algorithms and cyclic complexity. It's sad.

As for the author's distinction between "computer science" and "software engineering", well, I'm sorry but I really don't think someone who can whack and hack a server-side PHP application is a "software engineer". A software engineer, by my definition, is someone who owns and has read Knuth's "The Art of Computer Programming", knows who Edsger Dijkstra was (and can pronounce his name correctly), can define the difference between a binary and counting semaphore, and can tell me why Java and C++ are generally bad ideas for use in high-reliability, hard real-time systems. A real software engineer, in my opinion, should also know the difference between a waterfall and a spiral, and what IEEE 12207 is (DO-178B is a bonus, but I don't hold out much hope for ever seeing that on the resumes I get).

Most of the current crop of "computer science" graduates I encounter can't do these things to the level necessary for me to feel good about turning them loose with code that will eventually end up millions of miles away from Earth. They typically had the kind of college cirriculum espoused by Mr. Zambonini.

What I think is seriously lacking in college cirricula is a sense of what is really important in the "real world". I don't think we need more Java programmers, OO languages are not the be-all, end-all that the compiler vendors and pointy-haired bosses wanted us believe, and knowing how to design the life cycle for a project and then implement it is a critical skill that I feel gets seriously short-changed by academia. I won't even get into why the phrase "unit testing" drives me insane. We need people who can do real software engineering with the same level of rigor found in mechanical engineering, electronics engineering, or aerospace engineering.

For example, would you get into a car and drive from New York to Los Angeles if you knew the car was assembled using trial-and-error methods by people who couldn't tell you what was really inside the engine or transmission? I hope not. But, yet, that is how a lot of software is created. Is it any wonder that something like 50 to 70% of all major software projects either do not meet the original requirements, are buggy, or just flat don't work at all (see this month's (September '05) issue of IEEE Spectrum for some illuminating articles).

We need more rigor, more theory and more basics in our college courses. We need people who stand on solid theoretical ground, know where the theory came from and why, and have an appreciation for how that theory is applied in the real world. What we don't need are more web page designers, Java hackers and XML wizards.

Want another opinion? Try this one () from Joel Spolsky. Or, how about this comment from Alan Kay, one of the founders of OO programming: "Iíve heard complaints from even mighty Stanford University with its illustrious faculty that basically the undergraduate computer science program is little more than Java certification." (ACM Queue vol. 2, no. 9 - Dec/Jan 2004-2005 () . Need I say more?

2005-09-11 11:08:36
MISSING LINKS: We're hurting for real Software Engineers
I just discovered that the posting SW here isn't smart enough to handle embedded links. Here they are:



2005-09-11 11:09:46
Writing code + Writing docs
Honestly, the course isn't particularly important. No matter what you study, it will be obsolete in 2 years and the majority of stuff you will do your career will be something you have to learn anyway. Programming is in a constant state of flux.

The most important thing is to write code. Lots of code. And I'm not talking about toy programs either. By the time you get to your 3rd and 4th year, you should be writing complicated software that requires weeks of work and collaboration with at least one or two other people.

Which leads into the perennial problem with writing software. No one ever learns to write documentation. The amount of near illiteracy you see on public forums is staggering. If you can't describe your code in a natural language (e.g. english) then you will never be able to collaborate on a complicated project well.

I've seen too many fresh-outs who have never taken much more than classes that required writing toy programs in Java. I think Java is a significant advance for enterprise applications but I'm afraid it is a complete disaster for undergraduate studies in programming. I would hate to see even more emphasis on a single language--especially one as limited as Java.

2005-09-11 11:41:13
Missing the boat...
I have had this discussion often in my school and I think that most of the time when people are thinking of Computer Science they are really thinking about Information Technology or maybe even information science.

Before you continue, please note that I don't hold a degree other than CS in any low regard, rather that I wish there could be some standard brought to bear.

Computer Science is NOT a business degree. It isn't a trade school, programming track, or software engineering track. It is a SCIENCE program. In most schools, in fact, CS grew out of the Math Department. Computers USED to be for doing Hard Math Fast.

IT, or MIS (Management of Information Systems in my school) are degrees under the BUSINESS SCHOOL. They are fine degrees and they teach many great things for those that wish to learn them. But 99% of the time, the reason why those folks have the great high-level tools that they have are because some Scientist worked long and hard to develop an underlying protocol/tool/process/engine that made it possible.

Too often, though, kids enter a CS program because they 'like computers' and 'want to create games' and 'love 3D modeling.' A lot of these kids HATE MATH and dread the year of calculus required in our program... they have no desire to learn about language processing or the importance (and amazing power) of statistics and linear math in solving real-world problems...

I think it is great that you would like kids to graduate with good OOPS skills and great writing skills, etc. But you are asking that scientists get skills to help them in IT and that is not the right approach. If you don't want a SCIENTIST... then do not hire one. Hire an IT prof or MIS person... that's what they went to school for.

CS is FOR people who want to develop better AI, Vision Systems, HCI systems, optimized compilers, faster rasterizing engines, figure out faster factorizations, develop clusterzable apps, increase compression ratios, optimize multi-threading.

Let us develop those tools so that your code monkeys can write faster Point-of-sale applications and bigger web-based database front-ends.

Oh, you can't be bothered with that stuff? Great: while you are making $50k a year tweaking your UML/ER Diagrams I'll be building the next generation tool that industry will pay millions for.

I am not saying that what you aren't doing isn't important (or that it isn't good) it just bores me to tears and you can have it.

2005-09-11 11:43:09
What would you put in a Software Programming curriculum?
I think the classic mathematics- and theory-based "computer science" as a track in school is fine.

The problem you're seeing is that "computer science" does not adequately prepare a student for a career in "software programming" OR "software engineering".

This fault does not lie with "computer science". Rather, with the education system, the industry, and everything that lies in between.

If you're a Comp-Sci graduate and you want to work in the mainstream software industry, you would benefit IMMENSELY by seeking out courses or books on the topics which Dan mentions. If you have a solid Comp-Sci background, and have covered 3/4ths of the topics in Dan's list, consider yourself eminently hireable in the market.

But the question now becomes: if you want to work in mainstream software, or if you're a company in the market to hire someone for such a position, is "computer science" a necessary prerequisit?

In other words, would you rather have someone straight out of trade school, having taken all of the courses on Dan's list, or would you rather a classically trained computer scientist?

The answer, I think, for many potential employees and employers is starting towards prefering the the trade-trained programmer. Why?

It may be simply because it is not effective for companies to (internally or otherwise) trade-train flocks of computer scientists, when they could potentially hire people who took the burden of trade-training upon themselves.

Or there may be other reasons I'm missing...

Anyway, it's an interesting topic of discussion.


2005-09-11 12:34:45
typical attitude...

I really don't see anything on the list Dan mentions that most CS graduates (aside from maybe health and nutrition ;)) havent had experience with or exposure too. Perhaps things are different in US University CS programs, but it seems like he is being selective with the list of courses that he deems as irrelevant without mentioning that most of them (AI topics, compiler engineer, etc.) are advanced undergraduate topics that are taken *after* basic core courses are completed. The math, basics of programming, software design and development methodologies, and database principles among others on and off his list are usually taught earlier in a students academic career. The basics learned early on, whether in the form of thinking paradigms or more practical skills are usually applied in these more advanced topics whether we all realize it or not.

As far as particular technologies like XML and JAVA, while there might not be courses with "XML" or "JAVA" in their titles, rest assured most CS graduates I know (myself included) have had to use these languages throughout their academic career whether they were formally taught them or not.

His opinion makes me wonder if the author has graduated from a CS program. When I first finished my undergrad and started looking for a job, I encountered a similar attitude among headhunters, HR-types, business-types, and even some techies who could no longer (or ever) see the forest in spite of the trees. Being a young, "inexperienced" person I actually started to believe them, that I didnt really have any relevant practical skills. But believing *them* didn't help *me* get a job.

After many frustrating experiences and trial-and-error, I realized that it wasnt that I didnt have relevant skills, but that there was a communication disjunct between academia and the professional world. On my resume I took the focus off of general descriptions of coursework I had taken and placed it on the particular skills that I had developed while taking various courses and working on various projects. In the early interviews before I had gotten my first job, I made a concerted effort to make it clear that I did in fact have relevant skills that would be useful to their organization and to explain how and why they were relevant.

Honestly, I don't know about others, but it's been my experience that the difficulty of most real-world technical problems doesn't necessarily lie in understanding the technical details or the various technoligies used, but in navigating organizational politics, communicating with others, and being able to slog through mundane tasks that benefit the bottom line.

Let's be honest -- if someone "knows how to handle three dimensional diffuse lighting vectors," then they will probably be able to "create a customer feedback form for an insurance company."

2005-09-11 12:51:23
My CPS Curriculum
I have a similar academic background and interest. I thought about sticking around for another major and minor in other fields (psychology, finance, anthropology, sociology) for similar reasons, but decided not to. I think my CS advisor had a little to do with that. I've been working for a couple of years now and am in grad school.

You'll be fine no matter what you do. Just change your attitude about what you think you are capable of and focus on what you have done that relates to what you want to do professionally and you'll be set. I think your mind will be as pliable if not more than the people you will probably have to work with. Use that pliability to your advantage. People that are complacent in a particular domain of problems are really at a disadvantage in the long-term when it comes to software in my opinion.

2005-09-11 12:51:30
Missing the boat...
Hey DaveCrist,

good answer.

I think school should give more theory real world. Experience will be teaching thru interships. It was design to be this way.

But one thing that I saw on my 13 years working with computer (and I did everything. From bad graphical design thru programming and system analysis) is that the best people that I met had a better base and theory background and people that just learn the pratical approach has a lot of problems to catch with new technologies.

That's my two cents...

2005-09-11 12:53:32
Missing the boat...
(just fixing) I think school should give more theory than real world experience.
2005-09-11 12:54:50
Missing the boat...

good luck with that next-gen tool there btw... ;)

2005-09-11 13:09:55
typical attitude...
that should be "technologies" not "technoligies" :)

2005-09-11 13:15:45
We're hurting for real Software Engineers
ooooo... I'm not an embedded software engineer, but you make me want to become one so I can work for you! ;)

2005-09-11 13:18:06
The problem is in the basics
they DO do this in at least some programs...
2005-09-11 13:26:59
I agree completely with your message, in spite of your supercilious presentation.

2005-09-11 13:34:50
Lack of "machine" and also more underlying science
1. Yes! They still teach this stuff in CS programs. I only graduated within the past few years!

2. This is EXACTLY how it worked at my alma mater and it seems to be the case at many other schools. A lot of time and energy (and money;}) was spent on these topics you mention as well as the design and analysis of algorithms in my program. I think part of the reason certain professionals share the opinion that the author expressed is in part related to this fact. The link to the real-world is not always apparent to all parties involved.

2005-09-11 13:43:30
Lack of "machine" and also more underlying science
Technical writing was a required course at my institution. After being in the working world and having some exposure to software specs or the lack there-of, I'm starting to understand why. I remember problems being much more clearly defined in school and the emphasis being more on solving as opposed to defining. It can be more challenging to identify or define a problem as opposed to actually solving it! Perhaps that's why it is overlooked so often by business partners or users.

Learning how to ask the right questions in the right manner comes in handy. For someone who is particularly bright or has never really been challenged, this can be a humbling experience :).

2005-09-11 13:50:23
I don't think you have any idea what you are talking about
There is a standard computer science curriculum for universities (in the US anyway) and it does not include AI, machine learning, or Bayesian networks. I can guarantee you that pretty much no major university requires these classes unless you are on some kind of "statistics/AI" specialty track. All of these topics are interesting, but they are electives. When I was in school there were 21 required CS classes of which 4 were CS electives. The core classes where on programming, math, Operating Systems, Compiler Design, Computer Architecture, Programming Languages, Algorithms, and stuff like that.

Given that math, machine learning, and graphics are really hard topics, and that XML is like a 3 hour seminar, I don't think that is an imbalance just because there isn't an XML class.

Part of the required classes is an introduction to compiler design in which the student will learn how to write a parser. So while a student might not know the actual acronyms dom and sax, they could write a dom or sax parser from scratch.

So yeah, CS programs teach Computer Science not Computer Programming, they teach Mechanical Engineering not Auto Shop, and I didn't have to take a class on Nutrition and Health (wtf?). Based on your ignorance, it doesn't seem like you have any actual knowledge of what a computer science degree entails, so your opinion is kind of irrelevant.

As an actual point of comparison, my company hired a CS intern from UC Berkely this summer. He didn't know SQL, servlet programming, and had no software development experience, all of which is pretty important for enterprise software development. He took all of a week and a half to get up to speed and he did a fantastic job (by the end of the summer everyone wanted him to drop out and come work for us, he wisely declined).

2005-09-11 18:33:08
Missing the boat...
DaveCrist said:
"Too often, though, kids enter a CS program because they 'like computers' and 'want to create games' and 'love 3D modeling.'"

Wanting to create games is not necessarily a bad reason to go into Computer Science. Trust me, game companies need people with good theoretical fundamentals, too!

2005-09-11 18:46:10
Off base!
I am a computer science professor at a regional state university in the U.S.A. I want to take exception with some of the attitudes expressed in
Mr. Zambonini's blog.

I think Mr. Zambonini is looking for the wrong thing in a graduate and potential employee. Like many employers, he is looking for a student who has been trained in the specific skills his firm uses and needs, and is not finding these skills in CS graduates. Such employers seem to think that university education is supposed to be about training. Sorry. It's not. Morons can be trained. Universities are about education.

Another problem with Mr. Zambonini's critique is that it assumes that all firms desire the same skills as his firm does. At my university, many of our graduates go into IS, some go into design of embedded systems, and others do AI programming for commercial robotic systems. Lots go into Web design. One of our graduates is even doing PL/I programming for a major insurance company. How does he propose that we make all these companies happy? Make our bachelor's degree program seven years long?

Only about the top 5% or 10% in intelligence can survive a CS major at any reputable university. If you hire a CS graduate, you are hiring intelligence. In four years, there is no way we can train a students in all the skills you are looking for, and, even if we did, by the time they graduate a good chunk of that training would be obsolete or inadequate. However, an intelligent CS graduate can learn virtually any skill you want, and in surprisingly little time, too.

By the way, at my university we also have majors in IS and IT. Many of these students have been taught the skills you have asked for, but they are generally not as smart as CS majors, and if you really are doing interesting and complex stuff, you will be happier in the long run with a CS graduate. If your usual university is not giving you the quality of graduate you need, I'd guess that the problem is not the CS major, it's the university. Try another one! By the way, it's generally true that the more "practical" is the education at a college or university, the less talented is the student body.

One more word about the value of intelligence: has it ever occurred to IT managers that the professors at universities never were taught the skills that their firms are looking for? How do you think I learned (for example) Python, so that I could teach it to my students? No one taught it to me; I taught myself. Similarly, smart graduates can teach themselves the skills you
need. All you need to do is hire intelligent, motivated graduates.

A word about writing: At virtually all universities, the English Department teaches the writing courses. Educational politics being what they are, this will not change. The problem is that virtually no one gets a Ph.D. in English to go teach college writing: they all want to study literature. That means that, when they become professors, they teach flowery expression, and consider clarity, brevity, syntax, and punctuation as secondary issues, issues they would rather not have to deal with. One way of helping this situation is to have "intensive writing" courses, with lots of writing, in the major department. In my department, several of our courses are set up to be "writing intensive". Unfortunately, in a systems analysis course I have time to mark up a paper that has bad English, but little time to teach good English.

I'm afraid it's true that companies will have to live with this situation and make the best of it. Either that, or GET INVOLVED with your local university. Many CS departments have boards of advisors consisting of members of the companies that hire their graduates. Volunteer for this: you'll be surprised what a difference you can make! At my university, our Board of Visitors has given us leverage with the administration that we had never had before, and it has helped us eliminate some real problems.

2005-09-11 21:05:10
Nothing wrong with CS
I agree; there's nothing wronge with the CS program in its current state. I graduated in 2004 with a BS in CS, and every topic that has been covered about all I have studied. However, I do believe if you want to be a good programmer, CS is a way to go. I've learned through the CS progam how to build (program) an operating system, compiler, database, networks, etc......
2005-09-12 01:48:59
Wow, controversial!
Thanks to everyone for taking the time to post here.

I've only just noticed (after filtering through my hate-email!) that this was mentioned on Slashdot (how come the people in agreement post on Slashdot, and only those that think I'm nuts post here???).

Anyway, for those that have come straight here, there is more discussion on this topic on the Slashdot entry:


2005-09-12 05:15:11
Time for a separation?
I think it is high time to make a critical and crucial differentiation. IMHO.

The same way that most colleges have a science department and an engineering department. If we take a look at how the two department differentiate themselves, is that the sciences takes a theoretical look and strives to understand. While the engineering department takes theories and knowledge, and create and innovate novel ideas to apply these theorems to everyday lives and makes them useful.

If we continue this line of thinking, is it not time that we start having a new degree or course? Something like Software Engineering courses and Computer Engineering courses? Although I think Software Engineering would be better. I think SWEBoK (http://www.swebok.org) would be of great help here. These courses would have Requirements Analysis, Object Oriented Analysis and Design, Frameworks, Design Patterns, Development Methodologies, Testing Methodologies, Documentation, HCI, etc. as part of their offerings.

I feel that we still need the curriculum of computer science department intact. The same way that we still have the science department, so that we can continue to develop new and revolutionary ideas.

Any universities or colleges out there up to the challenge? We might be in one of the very important crossroads in the field of Computers. Time to take it to the next level.

2005-09-12 09:25:15
I mostly agree
Itís great to see such an interesting discussion! I would be very interested to see the demographics of the people who have responded. From the emotion in some of the replies it seems as if many people have taken this very personally - possibly CS undergraduates, CS graduates or people responsible for creating/delivering a CS course in a uni? My interest isnít particularly with these people, however I am extremely interested in opinions of fellow employers.

As the person responsible for the management and recruitment of developers in a UK based software development company, I share some of Dan Zamboniniís thoughts. For a long time I have wished that the graduates we have recruited were more commercially aware (find me a manager or project manager who works with developers who doesnít agree!)

In fact over the years I have become so disappointed with the quality of most of the graduates that I encounter that, these days, I rarely recruit graduates until they have some commercial experience under their belt. I also know that we are not the only company to think like this, so perhaps rather than personally attacking the author, students and universities should take this as valuable feedback from an employer Ė the person you should be out to impress!

Ultimately a university has a responsibility to its students to ensure that they are educated to a level that makes them employable. And students have a responsibility to themselves to study a course that makes them employable. If enough employers think like this then it could be the beginning of a problem. It is certainly something that students should think about before choosing which degree to do and within that, which optional courses to take.

There is a wealth of different computer based courses for student to choose from. I definitely see more CVs from CS students. Perhaps more effort is needed to advise them which course to take for their chosen career.

Personally I subscribe to the Joel Spolsky way of thinking in recruitment and look for are people who 1) are smart and 2) get things done (http://www.joelonsoftware.com/articles/fog0000000073.html - itís an interesting read) rather than skills based recruitment. That perhaps is even more worrying for CS students as, if I am not specifically interested in skills, and Dan Zamboniniís thoughts above are not valid (as so many people have claimed) then Iíd probably prefer to employ a Physics graduate (intelligent, great logical/problem solving skillsÖ).

The point raised below about university lecturers having to teach themselves the skills before they can teach their students is a good one, but also another worrying one. Perhaps there is an argument to bring in outside expertise for some of the skills training?

I dare say that the syllabus for the CS course varies wildly between the US and UK. I cannot personally speak for the US but know that the within the UK the content and quality of the courses varies enormously. If a few of them started taking these point on board then they may find that a larger percentage of the graduates they churn out are useful to the employers.

2005-09-12 10:54:46
Off base!
Thank you for your message professor. Especially for this comment, "Only about the top 5% or 10% in intelligence can survive a CS major at any reputable university. If you hire a CS graduate, you are hiring intelligence."

I believe next to the Physicists (mad geniuses), that us Comp. Scientists work the hardest. The Com. Engineers have it pretty tough too.

I imagine that the guy writing this weblog has a degree in management without any real technical background. What he doesn't realize, if a person is able to learn "neural networks, computer vision, artificial intelligence, robotics, compiler engineering, machine learning, quantum computing, Bayesian networks, embedded systemsÖ,"

that the subjects in his trivial list are just second nature to us. I think that this guy, like most other non-computer science IT professionals, is probably jealous that he can't make it in a class like artificial intelligence.

2005-09-12 11:18:54
Advanced degree
This is what the advanced degree is for. Typically, there is little if anything of value in the real world to be gleened from any course taken during the undergraduate studies. Only at the master's level will anything practical actually be taught.
2005-09-12 12:06:21
Re: Off base!
I don't like rising to flame-bait, but I can't help this one... I (the author of this weblog entry) studied a masters degree then a PhD in astrophysics.
2005-09-12 12:41:28
I mostly agree
i'm interested in demographics too. what is the name of your company?
2005-09-12 13:06:27
Off base!
Among all the responses here for this article, I think this one is the most right-on!!


2005-09-12 14:09:49
I mostly agree
I think this article is right on the money. I work in a big college in Florida (50,000+ students) and I run a fairly successful Cisco networking program. About 50% of my incoming students already work in the field, about 30% get jobs in Cisco networking before the end of their third class (CCNA), about 15-17% don't look for a job because they are concentrating on school, and the other 3-5% couldn't get a job to save themself (they can't pass drug screens).

When I came here the curriculum had 6 computer classes and 17 classes in business, accounting, finance, and general education (the associate's degree). After conducting research with area employer's we changed all that around. The students now get A+, MCP, Linux+, one or more programming classes in Java, C++, or .NET as "general ed of computing" and then specialize with 7 classes in CCNA/CCNP and two in VOIP. It seems to be working very well.

My colleagues from around the state are starting to see the value in a general computer background with a specialization field like Cisco, Microsoft, Unix, Linux, CISSP, and programming.

With respect to computer security classes I just gave a present on "why computer security courses are failing in community college" What was meant to be an intellectual discussion turned into a verbal bashing of "how could I ever have suggested such things..." like (1) no upward track to computer security training (where is entry-level), (2) no pay for good teachers ($35k is "average" starting pay for community colleges) (3) No one wants to hire someone with an associates degree in computer security, unless mom owns the company, (4) declines in programming courses are making it difficult to keep "good" security programs, (5) Too many non-computer security trained managers running computer security departments, and (6) the "best" computer security students are strong in mathematics...90% of incoming community college students cannot even place into college algebra...

As far as transfering in to a university...more and more, we are hearing employers scream about generalists with lots of business and management courses. They want a "chameleon," someone with a broad background who can adopt quickly in any circumstance...just make one type a programmer, one type a networker, and one type a databaser (if you will). Of course every employer has their own specific wish list too.

Very intriguing article. Kudos.

2005-09-12 17:02:18
The value of CS, and a rant on pedagogic calculus
Hmmm. Several professorial posters have justified their difficult, esoteric classes by arguing that the student that can survive them can easily learn anything the real world throws at them. One could also state the same thing for learning Chinese. It's quite difficult, and those that do well in it find computer languages easy. Just an observation I've made over the years. However, they are right, in a way, as my rambling below will touch on. But first, my anti-pedagogic calculus rant

Many sciences require Calculus. It's rather difficult, especially for math phobes and is a great way to cut down on the number of candidates for a popular major. The text books in the subject are opaque (compare any but Thomas to Thompson's 1910 "Calculus Made easy" if you doubt me), the instructors (usually foreign) are much more interested in completing their degree rather than dealing with these dumb Americans, and the professors (at least in my acedemic career) are incapable of conveying the excitement of what can be accomplished with Calculus (Refer to Darbyshire's "Prime Obsession" for a glimpse of how really cool it is). I'm sorry to say that you will not find this in freshman calculus 101. Most of you acedemics should just have the honesty to state that the course exists to get rid of those who cannot understand mathematics while being bored out of their skulls.

What a terrible fate for a beautiful set of ideas.

I've worked as a programmer for 25 years. Programmer is too narrow. What I really do is manage complexity. I've written code in assembler, C, Forth, Pascal, Lisp, Basic, Fortran,
Python, Java, Smalltalk... I learned all of these from the manual. I have used calculus once, (to develop an algorithm that took the number of rotations of a tape capstan and turn that into a linear measure of the tape used.)

That's the only time I've use it professionally. Now, in decades of lunchtime talks about physics with colleagues, I've integrated and differentiated countless times. I'm glad I know it, but it hasn't been much direct use in my paying work. However statistics, fourier analysis, and an host of other algorithms has been, and understanding these requires calculus.

What is directly professionally useful is also ephemeral. How to pass arrays in perl. How to make paintComponent work in Java Swing upon program initialization. How to set up a jar file, or a CLASSPATH. These might well be as useful 10 years from now as using common blocks or setting up print/format statements in Fortran is now. Rather silly to base a college course on them.

So. What should colleges insist their CS graduates study?

Analytic skills
Well taught calculus so you can understand statistics, which is very useful.
How computers work. And, or, and nor gates. adders. 2's compliment. Assembler. Pipelining, caching, networks, both serial and parallel. Network architecture. Interperters. Complilers.
With this information, you'll understand how to go about optimizing a system.
Languages. The Algol family. The Lisp family (forth, for extra credit). Now you have an idea of how to match a language with a task. Or you can avoid dogs, like C++ or PL-1.
Object oriented programming. I've only seen it done correctly once in my career. Learn when to use it (hint: big projects) and when not.

Algorithms. As a scientific programmer, I've had Numerical Recipies in C as a constant companion. Learn how and when to use each one.
You might only use 10% of them in real life, but you never know which ones will comprise that 10%.
Oh. You'll need quite a bit of mathematics above the calculus level if you want to this, I might add.

Code, debug, and code some more. Learn Unix, for serious projects, and windows because it's so prevalent. Write neat programs that do what you want them to. Games, simulations, AI, control robots... you imagine it and make it happen. Lots and lots of work. Be sure you enjoy it!

English. You have to be good at english to write good code. As mentioned by other posters, a class in literature won't cut it. I suggest a course in english taught by the journalism department. This is very important. I've thrown out reams of poorly documented code in my career, not because it didn't work, but because it was so poorly organised and commented that it was easier to rewrite it from scratch.

Computers are a lifetime study, a CS degree is only the start. Not to worry, however, because it's really fun teaching these dumb machine to appear be much smarter than they really are.

2005-09-12 23:58:15
One view of curriculum issues
In a paper in ACM SIGCSE conference in 2002, Bob Kummerfeld and I discussed a curriculum for educating software developers, within the constraints of a major in a typical US liberal-arts degree. This includes courses on a range of process aspects, but especially concentrating on "code construction" as represented in wonderful books like McConnell, Hunt-Thomas or Kernighan-Pike. See http://doi.acm.org/10.1145/563340.563367
2005-09-13 10:10:45
Great article
I graduated from Purdue in 94. They have multiple schools in the University which teach a computer curriculum.
School of Science - CS
School of Business - MIS
School of Engineering - Computer Engineering
School of Technology - Computer Technology (CPT)

I started in CS which is exactly how the Dan describes the courses. After 2 years of wondering why I hated my computer classes, I switched to CPT. I was saved.
This curriculum had classes that made sense. The freshman classes were basic, writing reports from a database. Iím dating myself in COBOL. By the time I graduated they switched to Delphi and VB for freshman classes, I think they use Java and .Net today. Other classes they taught freshmen were Word, Excel and Access. Nice to know that your future IT department would know how to use a word processor. In Junior and Senior level classes the classes focused in SQL databases, Network Design and Application design creating Data Diagrams and ERDs. My senior classes I was taking a class in VB3. My first job I was using VB and Oracle.

2005-09-13 15:43:44
Not quite
Dr. C does a mostly good job of explaining the virtues of a CS curriculum. It's a shame he stooped to trashing English teachers in the process.

I went into IT in 1995 after having bounced between math and English departments at various colleges since the age of...well, since 1974...and when Dr. C says English professors "teach flowery expression, and consider clarity, brevity, syntax, and punctuation as secondary issues, issues they would rather not have to deal with", he is, in my experience, dead wrong.

I have never met a composition teacher of whom that is true.

In fact, I've never met a composition teacher who approved of "flowery expression"--flowery will get you redlined in freshman comp and laughed out of creative writing. The book we used in both my first creative writing course and my survey of poetry is the brilliant (now overpriced) Sound and Sense by Laurence Perrine, who has this to say in chapter 15, "Bad Poetry and Good":

"Rhetorical poetry uses a language more glittering and high flown than its substance warrants. It offers a spurious vehemence of language--language without a corresponding reality of emotion and thought underneath. It is oratorical, overelegant, artificially eloquent. It is superficial and, again, often basically trite. It loves rolling phrases like 'from the rocky coast of Maine to the sun-washed shores of California' and 'our heroic dead' and 'Old Glory.' It deals in generalities. At its worst it is bombast. In this book an example is offered by the two lines quoted from the play-within-a-play in Shakespeare's A Midsummer Night's Dream:

"Whereat with blade, with bloody, blameful blade,
He bravely broached his boiling bloody breast.

I will grant that creative writing teachers are typically tolerant of unusual syntax and punctuation, if they serve the purpose of the writer, but even that has its limits.

I never took a class from this guy because I was friends with him and his family: http://www.arktimes.com/Articles/ArticleViewer.aspx?ArticleID=b732eb20-4231-403d-a8fb-678f726c69e6 , but I do remember walking down the corridor of Kimpel Hall with him and trying to express my love and admiration for Kathy Acker's Blood and Guts in High School.

The book has a lot of power just from Acker's command of language, but she does more. She's got hand-drawn maps of characters' dreams in it, an Arabic grammar that tells a story, a tour de force passage in which two poems are interleaved, line by line, and are coherent narrative read separately or together. It's a piece of experimental writing which, atypically, works.

I remember Bill's response: "So, it's got some funny stuff in it?" He didn't mean comedy.

Math or English (and maybe philosophy, theology, or history) will prepare you for a complete, well-rounded life--personal, professional, and civic--better than any other undergraduate curriculum.

P.S. The current eleventh edition of Perrine is priced at about twice what it's worth, so if you see an old edition cheap, buy it as a gift for any aspiring poet, young or old.

2005-09-14 18:46:41
Re: Off base!
So by the standards implied by your blog entry, how can you possibly be an authority on computing? Your university didn't teach you how to deal in all the acronyms listed in your bio, right?

Or did you maybe develop the skills and knowledge base there that allowed you to learn it on your own later?

2005-09-16 06:56:59
You are missing the point
I think you are missing the point. You study CS because you are interested in Computer Science. If you just want to learn some basic programming and such, you study Software Engineering or whatever instead.
For those of us that has a CS degree, we simply dont need to study stuff like XML at uni since we can learn the API:s in an afternoon or so.

Your arguments are a bit like "why do neurosurgeons need to study all this internal medicine stuff, give them a circular bone saw and a scalpell and let them practice instead."
As a computer scientist, you should not just know the way things (languages, tools, technologies etc) work and look but focus on why they are built the way they are and look for better ways to do things. And this no matter if you just work with creating a web form for an insurance company.
It is quite obvious that you do not have a CS degree yourself, or you would know that 80% of the stuff you learn is not technology related but the basis for all computing (maths) and thinking critically about things. Doing that you are always looking for ways to do things better and in a smarter way.
So the problem is not the contents of the CS education, we will always need front-line competent folks that doesnt fall for market hype but strives to take the CS discipline to new heights, while we will also need people creating web forms for insurance companies, which you do not need a CS degree for, even though you are more likely to do a better job if you have one.

2005-10-06 16:10:50
SE is cool, CS is good ;-)
I've got a Bachelor of Software Engineering and I'm doing fine (actually I'm happy as a team leader) with my current job. Nevertheless, what I think is that even if I've got an CS, instead of SE, and apply for a development job then it may take me a bit to face with the industry (such as development life cycle, A&D etc.); but I can easily get over it with the solid knowledge offered in CS courses. The fact is that currently I have to train myself on the knowledge that I never learnt in SE such as parsing, compiler etc. and sometimes look down on myself (just a few seconds ;-) ) because I cannot understand some articles discussing AI.

The understanding of "industry" strenghts (and hypes ;)) such as XML, SOA, SAX, DOM is good but I don't think it's that too difficult. Paradigms such as OO, patterns etc. are also good. But it's better if you have understood the principles. For example, if you understand the design principles then you don't even need to "memorize" the so-called design patterns ;-).

To CS students who are worried that the knowledge you have cannot get you a development job, my advice is: don't worry, you are VERY well prepared, pick a book or two in the subject that your "dream" employer wants (i.e. Java, .NET, SOA, A&D, patterns etc.) read it, experiment it and you'll be fine (of course, you have to learnt more after being employed ;) ).

When diving yourself into this fast pace ever-changing field, either you're computer scientists or software engineers, you should never stop learning the things that you NEED for your job. And I believe that a good foundation knowledge will do you more good in learning new things.