1. Assessing the Effectiveness of Individual Reflections on Video Feedback
Walter W Schilling, 2021 ASEE Annual Conference. Online., (July, 2021)
We know from research that feedback to students is an decisive aspect in the learning process. Students learn better when they receive relevant and timely feedback from faculty members regarding their assignments. Multiple studies have shown this. However, if students do not review the feedback, it is not effective, and faculty members routinely speak to anecdotal stories of students disregarding feedback given to them.\p In previous papers, the usage of multimedia feedback has been discussed. In essence, with multimedia feedback, traditional written comments are generally replaced with a short, narrated video whereby the feedback is provided both using audio and visual techniques. Overall, this approach has been shown to be quite effective for communicating with students. However, as with traditional feedback, the videos are only effective if students watch them.\p This paper will present a new approach toward video feedback, namely integrating an optional individual reflection into the process. Students who watch the video and complete a brief reflection can make back some points toward their assignment. The paper will show comparisons of student performance across multiple sections using this mechanism, comparing the students who did not generally watch videos, the students who watched videos but did not submit reflections, and the students who watched videos and submitted reflections. Results will be provided across multiple courses and multiple sections whereby this approach was employed.
2. Work in Progress: Integrating the Entrepreneurial Mindset into a Software Requirements Course
Walter W Schilling, 2020 ASEE Annual Conference. Online., (June, 2020)
One of the most challenging aspects of software engineering is teaching students requirements elicitation. Software requirements elicitation requires complex thinking and a thorough understanding of the customer and business needs. Traditionally, requirements elicitation courses have focused on pure documentation of requirements, the focus being on drafting unambiguous statements properly formatted to follow an IEEE standard. However, the challenge of requirements elicitation is often not in the documentation of requirements, but rather in understanding the needs of a customer.\pThis work in progress paper intended to provide a case study of a novel approach to integrating the entrepreneurial mindset into a software requirements course. Working in teams, students are given an extensive scenario related to a real-world medical issue introduced by a brief video. Through the remainder of the course, students interview other students, real world practitioners, and others to understand the value of the product and the needs of potential clients before drafting a final requirements document which then could be used to develop the project. Through this approach, engineers communicate with nurses, athletes, pharmacists, and other non-engineers, learning the skills of teamwork, the perspectives of non-engineers, the limitations of technology, and in some cases, learn that a project that seems advantageous may actually not be successful. The paper will describe the project, the materials created for the project, and provide student observations on the success of this approach.
3. WIP: Lessons Learned from Applying Standards-based Grading to a Software Verification Course
Walter W Schilling, 202 ASEE Annual Conference. Online., (June, 2020)
One of the newer approaches to grading and assessment is standards-based grading. Standards Based Grading, otherwise known as SBG, directly measures student’s proficiency in a course based upon specific course learning outcomes. This approach offers an alternative to traditional, summative based grading systems. Thus far, there have been very few attempts to integrate standards-based grading into computing fields. This work in progress paper will discuss the process of how SBG is being integrated in a sophomore level software verification course. The paper will include a listing of the specific concepts assessed in the course as well as a discussion of the mechanisms used to provide feedback to students on their performance. The paper will also provide a discussion of the challenges of integrating SBG into a college course, and some of the reasons why a complete SBG approach has not yet been undertaken for the course.
4. WIP: Active Learning Exercises to Promote System Performance Testing
Walter W Schilling and Brad Dennis, 2017 ASEE Annual Conference. Columbus,OH., (June, 2017)
The verification of system performance is a major aspect necessary to ensure the proper operation of software systems. In numerous, high-profile cases, deployed systems, such as e-commerce sites and the Healthcare.gov website, have failed due to performance related issues. The IEEE Software Engineering SE2014 document encourages software engineering programs to cover multiple forms of testing, ranging from unit tests through performance tests. Unit testing, at one extreme of this range, is easily taught as it focuses on small scopes and detailed functionality. Another type of testing, integration tests, can also be easily expressed based upon sequence diagrams. These two items must have deep coverage per the curriculum guidelines. However, at the other end of this spectrum is performance testing. Performance testing is much harder to teach, while also needing much less coverage per the IEEE SE2014 guidelines. This article will define a set of active learning exercises which were developed to aid in the understanding of performance testing. These exercises allow students to execute performance tests and analyze the results, while not requiring students to develop any performance tests. In each case, the performance behaviors are readily traced back to design and implementation decisions made which significantly limit the scalability of the systems under test. This article will describe our experiences with implementing the exercises, as well as provide feedback from students who have used the module to learn about performance testing.
5. Infusing Software Security in Software Engineering.
Sushil Acharya and Walter W Schilling, 2017 ASEE Annual Conference. Columbus,OH., (June, 2017)
Software is now ubiquitous and software security is now realized as a growing threat. It is important for software developers to fix software security problems, however more imperative is for software developers to understand that security features are not to be introduced as patchwork when a security situation arises but are to be addressed and handled very early in the software development lifecycle. Industry’s general lack of ignorance of software security benefits and more importantly the shortage of software practitioners possessing software security understanding creates multitude of problems in the software industry. Imparting real world experiences in the academia as well as the industry is a challenge due to lack of effective active learning tools (ALT). Riding on the success of developing and disseminating, 42 delivery hours of active learning tools in the area of software verification and validation the authors propose to partner with industry to develop 14 delivery hours of course modules developing ALTs in the form of class exercises, case studies, and case study videos and delivering them using a flipped classroom model. Through a gap analysis exercise jointly carried out with industry partners a draft requirements list has being identified. Specific exercises are being developed using an iterative development methodology. Student understanding is proposed to be assessed through quizzes, exams, assignment, and a learning survey. Once developed the ALTs will be made publicly available through a website. This paper discusses continuing work on the gap analysis in software security education, presents proposed contents areas for ALT, shares structures of three developed/proposed ALTs, presents a sample of a survey instrument, and presents a sample ALT on case study video.
6. The Integration of Novel Forms of Feedback into Software Engineering Courses.
Walter Schilling, 2016 ASEE Annual Conference. New Orleans, LA., (June, 2016)
Software Engineering students exhibit a wide array of learning styles across the perception, input, organization, processing, and understanding dimensions. To improve students’ performance in the classroom, many techniques have been developed to address these variances. Active learning has a long history of usage in the software engineering classroom, and the disciplines strong history of diagramming and visual representations have been very supportive to the large percentage of students who are visual learners. Coaching by faculty in project based courses also is common in the discipline. However, when it comes to providing feedback to students on formally submitted assignments, the main method employed is still written comment, which is not conducive to visual learners. Written comments are embedded in source and marked on design diagrams, using annotations or colors to distinguish them from the original work. This method is most prevalent in the community because overall, it is the simplest form of feedback that a faculty member can provide to students. However, written feedback is often highly ineffective at improving student performance, as many students simply do not read the comments because the students feel they are not relevant to their performance. This survey paper presents an assessment of an alternative methods of providing feedback to students using audio-visual techniques. In lieu of written feedback, students are provided feedback for software engineering exercises through the use of short multimedia files. The article describes the pedagogical foundations for the technique, specifics of the technique used, student perceptions of the technique, and an assessment of the learning gains from using such a method across several software engineering courses. In general, students are shown to prefer the technique versus traditional grading, and an improvement in overall outcomes for the course is shown to exist as well.
7. Enhancing Verification and Validation Education Using Active Learning Tools Developed through an Academia-Industry Partnership.
Sushil Acharya, Priya Manohar, Walter Schilling, and Peter Wu., 2016 ASEE Annual Conference. New Orleans, LA., (June, 2016)
Imparting real world experiences in a software verification and validation (S/W V&V) course is a challenge due to lack of effective active learning tools. This pedagogical requirement is important as graduates are expected to develop software that meets rigorous quality standards in functional and application domains. Realizing the necessity of such teaching tools, the authors designed and developed eighteen (18) delivery hours of case studies, sixteen (16) delivery hours of exercises, and six (6) delivery hours of case study videos for use in courses that impart knowledge on S/W V&V topics viz. requirements engineering, reviews, configuration management, and testing. Four key skill areas sought after by employers, namely communication skills, applied knowledge of methods, applied knowledge of tools, and research exposure are used to drive the development funded by a National Science Foundation grant and perfected through an industry-academia partnership. In this paper we discuss in detail the four project plans the researchers and their industry counterparts followed in the past two years in the development and eventual dissemination of active learning tools. A course enhancement plan was used to drive activities related to reviewing, enhancing and modularizing modules identified by a gap analysis performed by focus group comprising of industry and academic partners. The course delivery plan was used to drive activities related to developing content delivery strategies. An evaluation and assessment plan was used to drive activities related to periodically evaluating student learning and assessing program. And finally a course dissemination plan was used to drive activities related to disseminating course modules and assessment reports. Through a workshop attended by faculty members form 11 universities the developed active learning is being disseminated.
8. Analysing the Impact of Viewership of Multimedia Feedback.
Walter W. Schilling, Jr., CIPTTE 2016 (Conference on the Impact of Pen and Touch Technology in Education), (April, 2016)
9. Analysing the impact of asynchronous multimedia feedback on novice computer programmers
Walter W. Schilling, Jr., 2015 Frontiers in Education Conference, (October, 2015)
For many engineering students, freshman programming represents one of the hardest courses for them to master. Unlike other science fields, few students are routinely exposed to programming in the K12 system. This can make the freshman programming course daunting. However, in the field of software engineering, success in this area is vital, as success in nearly all future courses requires mastery of this skillset. In the engineering field, we find that many students are visual learners. These students learn best by seeing, and they can perform very well in the classroom with the appropriate usage of teaching styles. However, when it comes to providing feedback to students on submitted assignments, the main method employed is the written comment, which is not conducive to visual learners. From a faculty member's standpoint, this makes sense, as it is the simplest form of feedback. However, written feedback is often ineffective at improving student performance, as many students simply do not read the comments because the students feel they are not relevant to their performance. This can be compounded in the freshman year, as students are still learning what is meant to be an effective college student. At higher levels, an alternative feedback mechanism, namely asynchronous multimedia feedback, has shown great promise. In lieu of written feedback, students are provided feedback for software engineering exercises through the use of a short video made via video capture. The video captures in multimedia format the instructor's perceptions and actions when grading a given assignment. The video shows, in real time, what the instructor saw, whether it is a program crashing or the successful operation of the program. Furthermore, it provides the instructor the ability to potentially fix simple blatant errors and see the instructor's debugging strategy. The article describes the pedagogical foundation for the technique, specifics of the technique used, student perceptions o- the technique, and an assessment of the learning gains from using such a method in an introductory freshman programming course. In general, students are show to prefer the technique versus traditional grading, and a statistically significant improvement in overall outcomes for the experimental course is shown to exist. A statistically significant correlation between the watching of videos and outcomes is also shown.
10. Case Study-Based Educational Tools for Teaching a Software V&V Course at the Undergraduate Level
Priya Manohar, Robert Morris University; Sushil Acharya, Robert Morris University; Peter Y. Wu, Robert Morris University; Ali A. Ansari, Virginia State University; Walter W. Schilling Jr., Milwaukee School of Engineering, 2015 ASEE Annual Conference and Exposition, Seattle, Washington, (June, 2015)
One of the critical problems facing software industry today is the lack of appreciation of the fullbenefits that can be derived from Software Verification and Validation (V&V) and an associatedproblem of shortage of adequately trained V&V practitioners. To address this situation, softwareV&V course curriculum is being improved at author’s institution via a NSF-funded project. Thebasic objectives of this project are to improve software education to make it well aligned withacademic research as well as industry best practices. In addition, it aimed that the course materialcould also be used to enhance on-the-job professional training in SW industry settings, therebyhelping to increase the pool of professionals with contemporary V&V knowledge and skills.The new course curriculum enhancement described in this paper is guided by academic researchand industry best practices that focus on four specific V&V topics: requirements engineering,reviews, configuration management, and testing. Among many educational tools that are beingdeveloped to achieve project objectives, the work related to the development of case studies isdescribed here. Historically, case studies have been as educational tools in business, law andmedicine but not so much in software engineering. The hypothesis is that case studies would beeffective educational tools to introduce real-world professional practices into the classroomwhich would help the students in identifying and solving problems, and develop a perspective onknowledge application. In this paper we describe a set of V&V related case-studies that we havedrawn from industry experiences and developed them as pedagogical tools. These case-studiescover several important topics in SW V&V domain such as legal issues in software, softwareconsumer protection, and requirements from the customers’ perspectives. We will also report onthe results of initial implementation of these case studies in the classroom.
11. Integrated Active Learning Tools for Enhanced Pedagogy in a Software Engineering Course
Sushil Acharya, Robert Morris University; Priya Manohar, Robert Morris University; Peter Y Wu, Robert Morris University; Ali A. Ansari, Virginia State University; Walter W Schilling Jr., Milwaukee School of Engineering, 2015 ASEE Annual Conference and Exposition, Seattle, Washington, (June, 2015)
Effective teaching requires effective teaching tools. This pedagogical requirement is especiallyimportant for software engineering education where graduates are expected to develop softwarethat meets rigorous quality standards in functional and application domains. To enhancestudent’s understanding of the needs of the professional software industry, lecture notes aresupplanted by additional pedagogical tools being developed at author’s institution for a softwareverification and validation (V&V) course. These active learning teaching tools consisting ofclass exercises, case studies, and role play videos are being developed by partnering with theacademia and industry. The basic objectives of the project are to improve software education tomake it well aligned with academic research as well as industry best practices. This project isbeing funded through a NSF-TUES grant.Through this project twenty (20) delivery hours of lecture slides, eighteen (18) delivery hours ofcase studies, sixteen (16) delivery hours of exercises, and six (6) delivery hours of role playvideo totaling sixty (60) delivery hours of Software V&V course modules are being created. Inspring 2014 diverse case studies were delivered to Software Engineering juniors in the author’sinstitution and their learning outcomes tracked using a survey instrument. The results show thatthe students were able to better comprehend V&V topics like requirements engineering, reviewsand inspections, configuration management, and testing. The knowledge disseminated throughlectures seemed to be reinforced by the case studies. By spring 2015 more course materialsincluding teaching tools like exercises and role-play videos are expected to be created anddelivered. The results of this research work will be shared during the poster presentation atASEE. As part of the project dissemination plan the teaching materials will be made available tointerested institutions and professional organizations. An invitation only workshop is alsoplanned for August 2015 to share developed course materials and delivery mechanism.
12. Work in Progress: Using Videotelephony to Provide Independent Technical Critique of Student Capstone Projects
Walter W. Schilling Jr., Milwaukee School of Engineering; John K. Estell, Ohio Northern University; Khalid S. Al-Olimat P.E., Ohio Northern University, 2014 ASEE Annual Conference, Indianapolis, Indiana, (June, 2014)
As part of ABET accreditation, students in engineering programs are required to complete acapstone project. One challenge that programs face is providing appropriate technical and professionalfeedback to students on their projects. Students may be working in an application domain in which thefaculty member has limited knowledge, or may be using newer technologies that the faculty memberhas not used before. To overcome these problems, it is often advantageous for the team to partnerwith an industrial mentor. The industrial mentor can provide technical assistance to the project as wellas provide impartial and unbiased feedback on the status of the project. However, this is not alwaysfeasible, as not all campuses are not located in urban areas, and stakeholders may not be local.This work in progress article describes the first part of a two part project to overcome some of theseproblems. In this project, capstone teams work directly with an alumni mentor on the project in anasynchronous format using videos and screen captures. As the students complete significantmilestones, deliverables are critiqued by an alumnus of the institution impartial to the project. Studentsreceive a short video with both written and verbal feedback on the deliverable, which can then beincorporated into future projects. This article describes the concept as well as provides a preliminaryassessment of the technique.
13. Collaborative Education: Building a Skilled Software Verification and Validation User Community
Sushil Acharya, Robert Morris University; Priya Manohar, Robert Morris University; Walter W Schilling Jr., Milwaukee School of Engineering; Ali A. Ansari, Virginia State University; Peter Y Wu, 2014 ASEE Annual Conference, Indianapolis, Indiana, (June, 2014)
Software quality is a crucial issue in software engineering. As software has become ubiquitous, softwareproducts have become critical. This poses a problem in the software industry, as there is generally a lackof knowledge of Verification and Validation (V&V) benefits and a shortage of adequately trained V&Vpractitioners. This NSF-TUES funded project improves software education at the undergraduate level andenhances on-the-job professional training, thereby increasing the pool of professionals with V&Vknowledge and skills. The project team consists of an academia-industry collaboration involving sevendistinguished academic programs and four industry partners. The project team is critically examining theexisting V&V course at xxxx University, enhancing and modularizing selected topics, and improvingdelivery strategies by incorporating academic research findings and industry best practices. The cogentmodules and strategies developed is being assessed, evaluated, and shared among project participantsand disseminated to other institutions through NSF supported centralized repository (NSDL),conferences, publications, professional societies, and websites. Students and practitioners are beingequipped with fundamental theoretical knowledge and invaluable hands-on-experiences that ismeasurably increasing their ability to contribute to the industry. This effort is being led by investigatorsand academic partners with significant experience both in academic and industrial settings includingsoftware development, course development, and project management.
14. Assessing the effectiveness of video feedback in the computing field
Walter W. Schilling, Jr., 2013 Frontiers in Education Conference, (October, 2013)
Engineering students exhibit a wide array of learning styles across the perception, input, organization, processing, and understanding dimensions. To improve students performance in the classroom, many techniques have been developed to address these variances. The computing fields, however, tend to have a large percentage of students who are visual learners. These students learn best by seeing, and they can perform very well in the classroom with the appropriate usage of teaching styles. However, when it comes to providing feedback to students on submitted assignments, the main method employed is the written comment, which is not conducive to visual learners. This method is most prevalent in the academic community because overall, it is the simplest form of feedback that a faculty member can provide to students. However, written feedback is often highly ineffective at improving student performance, as many students simply do not read the comments because the students feel they are not relevant to their performance. This paper presents an assessment of an alternative method for providing feedback to students: video feedback. In lieu of written feedback, students are provided feedback for software engineering exercises through the use of a short video made via video capture. The video captures in multimedia format the instructors perceptions and actions when grading a given assignment. The video includes both aural commentary as the assignment is assessed, as well as dynamic visuals of the grading process, demonstrating failures and improvements that can be made in the submitted assignment. The article describes the pedagogical foundation for the technique, specifics of the technique used, student perceptions of the technique, and an assessment of the learning gains from using such a method in a junior level class. In general, students are show to prefer the technique versus traditional grading, and an improvement in overall outcomes for the course is shown to exist - s well.
15. Instilling the Necessity of Lifelong Learning using Article Reviews
Walter W Schilling Jr., Milwaukee School of Engineering, 2013 ASEE Annual Conference, Atlanta, Georgia, (June, 2013)
ABET Criterion 3 requires students to demonstrate “a recognition of the need for, and anability to engage in life-long learning.” To successfully obtain this skill by graduation, it isimperative that students be allowed to practice these skills in all classes. This leads to achallenge: how can one effectively integrate and assess student’s lifelong learning skills inall classes in an efficient manner? One mechanism for addressing lifelong learning skills isthe research paper. While the formal research paper provides the students with manyimportant skills, especially for those who intend to go on to graduate school, the verynature of the research paper may diminish its effectiveness at promoting lifelong learningfor the vast majority of students. Following the traditional format for a formal researchpaper also rules out the usage of podcasts and webinars, two evolving formats forinformation exchange.This article presents a mechanism for helping students to learn about the importance of lifelonglearning through the usage of short, non-intrusive “Article Summaries”. Article Summariesallow students, in a non-intrusive fashion, to learn in greater depth about the material which isbeing taught in a course as well as explore the differences in style between various mediaformats. Through student assessment data, article summaries are demonstrated to be a light-weight effective mechanism for increasing student engagement in a course as well as instilling anappreciation of the need to continually learn.
16. Enhancing Student Comprehension with Video Grading
Walter W Schilling Jr., Milwaukee School of Engineering; John K. Estell, Ohio Northern University, 2013 ASEE Annual Conference, Atlanta, Georgia, (June, 2013)
Engineering students have different learning styles. Many techniques have been developedin the classroom setting to address these differences, and these approaches have been welldocumented. One of the more interesting approaches that has been recently developed isthe inverted classroom, in which students view short videos prior to lecture as one of theirpreparation activities. However, when it comes to providing feedback to students onsubmitted assignments, the main method employed is still the lo-tech paper-based writtencomment. This method continues to be employed because it is simple to do, can easily bedone offline, and is domain independent. However, written feedback is often highlyineffective; it is often the case that students will just look at their score, and then file thegraded assignment away in some obscure pile, never to be seen again. As a consequence,the time and effort spent by the instructor is wasted.This paper presents an alternative method to provide feedback to students, and represents anatural extension to the video component used in the inverted classroom. In lieu of writtenfeedback, students are provided feedback for their submitted computer programming exercisesthrough the use of a short video made via screen capturing software by the instructor. Thetechnique is relatively straight-forward. For each student’s submission, the submission is firstopened, and then the instructor places the screen capture software into record mode. Theinstructor then goes through the submission, providing verbal commentary as elements to bebrought to the student’s attention are visually brought to their attention. The submitted paperwill describe the specifics of the technique used, student perceptions of the technique, acomparison of faculty effort, and assessment from the use of this technique at two private 4-yearinstitutions located in the Midwest.
17. Teaching software inspection effectiveness: An active learning exercise
Walter W Schilling, 2012 Frontiers in Education Conference, (October, 2012)
This paper discusses a novel active learning exercise which teaches students how to perform and assess the effectiveness of formal software inspections. In this exercise, students are responsible for selecting an artifact from their senior capstone design projects. The students then use fault injection to strategically place faults within the artifact that should be caught by the inspection exercise. Based on the needs of the team, students prepare an inspection packet consisting of a set of inspection instructions, applicable checklists, and the inspection artifact. Students then “hire” a set of inspectors based on classmates' backgrounds and experiences. The team leader then holds two inspection meetings and reports the results. The results are then used to assess the effectiveness of the inspection. Overall, in analyzing 5 years worth of data from this exercise, it is found that students are capable of selecting appropriate materials for inspection and performing appropriate software inspections. The yield of students is lower than an experienced professional might have and the inspection rates tend to be slightly higher than desired for their experience. However, the yield is related to individual preparation time. Students overall find this to be a highly educational experience and highly recommend it be continued for future classes.
18. Effective Assessment of Engineering Design in an Exam Environment
Walter W. Schilling Jr., Milwaukee School of Engineering, 2012 ASEE Annual Conference, San Antonio, Texas, (June, 2012)
One of the most difficult aspects of engineering is the effective teaching of engineeringdesign. While it is paramount that every engineering student be exposed to engineeringdesign, it can be difficult to assess the capabilities of individual students. Most designassessment is typically conducted at the project or team level, and many assessments ofdesign effectiveness only rely upon the capstone experience. This is purely inadequate.Instead, what is needed is an effective method that can be used to assess the design skills ofindividual students in an exam setting.This article will discuss an approach to assessing design skills in the exam environment. Itallows for the effective assessment of design skills and practices in an exam environment withoutplacing undo mental stress upon the student. The approach involves a short case study providedto the students as part of an exam review sheet, the construction of design questions based uponthe case study, the construction of detailed rubrics to assess the quality of the design, and theadministration of the exam in a controlled setting. Student achievement is discussed, as well asthe advantages and disadvantages of this approach to assessing design skills.
19. Effective Active Learning Approaches to Teaching Software Verification
Sushil Acharya, Robert Morris University; Walter W. Schilling Jr., Milwaukee School of Engineering, 2012 ASEE Annual Conference, San Antonio, Texas, (June, 2012)
Course based exercises that map theory to practice with active learning lead to increasedunderstanding and knowledge retention. This is vital within the field of software engineering, assoftware engineering is an applied discipline. Software Verification is a software engineeringprocess that ensures the software product is being built correctly and is considered an industrybest practice. Teaching verification to undergraduate students requires additional coursematerial development and preparation in terms of the verification theory to be covered, thedomain to be used, and the exercises to be performed, as certain verification topics are timeconsuming and certain codes are not suitable for verification exercises. This paper looks intothe approaches taken by two institutions to teach verification using the active learningapproach. An overview of the existing literature is provided, including the pros and cons of theapproaches. This is followed by an overview of the active learning approaches to softwareverification from two different universities, and an analysis of the pros and cons of eachapproach. Also included is an assessment of student learning at each institution.
20. Teaching Software Engineering: An Active Learning Approach
Walter W. Schilling Jr., Milwaukee School of Engineering; Mark J. Sebern, Milwaukee School of Engineering, 2012 ASEE Annual Conference, San Antonio, Texas, (June, 2012)
Software Engineering is a core component of many computer engineering programs. Insoftware engineering courses, students are taught to apply their programming anddevelopment skills to solve a larger scale problem. The resolution of this problem involvesthe development of an understanding of the problem as well as an analysis of solutionalternatives. Unfortunately, in many cases, the software engineering course is offered latein the curriculum, typically at the senior level. This makes it difficult for students to applythe knowledge that they have learned effectively outside of the course on capstone andother academic projects. Students often comment that it would have been “nice to knowthis” before making the wrong decisions on their capstone projects. Thus, to be successful,components of software engineering need to be taught earlier in the undergraduatecurriculum. This shifting to an earlier level, however, poses pedagogical issues.This paper describes the metamorphosis of an undergraduate software engineering course from asenior level course to a sophomore level course. In this course, students are taught to usesoftware engineering tools and practices in pursuit of a solution to a software based embeddedsystems problem. Students actively work together in teams while theoretical softwareengineering concepts are delivered using “Just in Time” instruction.In addition to providing an overview of the course material and exercises, this article will discussthe changes made to the course in each of the previous 4 offerings. Changes were based uponstudent comments and other feedback. An analysis of student comprehension and performanceagainst course outcomes will also be provided.
21. Teaching Software Security: A Multi-disciplinary Approach
Walter W. Schilling Jr., Milwaukee School of Engineering; Eric Durant, Milwaukee School of Engineering, 2012 ASEE Annual Conference, San Antonio, Texas, (June, 2012)
As computing devices become more and more ubiquitous, the importance of softwaresecurity cannot be overlooked. As such, many software engineering and computer scienceprograms offer an elective course in software security. While the title of these courses isoften similar, the content is often vastly different, reflecting the significantly large aspectsof software security. Certain aspects of security appeal to practitioners, certain aspectsappeal to Computer Scientists, and certain aspects apply MIS personnel.In order to provide a holistic view of computer security, software engineering students need tohave exposure to all three aspects. Thus, for software engineering students, a single course insecurity can be inadequate. To combat this problem, the has developed athree course sequence in software security targeting the multi-disciplinary problem of security.While each of the three courses addresses software security, each course targets a differentaspect. An Introduction to Network Security offers students an understanding of the nature ofnetwork security. Secure Software Development focuses the design and construction of softwaresystems in a manner in which security is built into the product from the beginning ofdevelopment. Information Security offers students an understanding of the techniques used toensure that data and other systemic information is protected using the most appropriatetechniques. Since the development of this sequence, one or more of these courses has been takenby approximately 100 students in the software or computer engineering programs at . This article provides an overview of the three courses offered at , the challenges of offering these courses as independent electives, and student impressionsof the security course series.
22. Faculty Usage of Cloud Computing: The Capabilities and Risks
Walter W. Schilling, Milwaukee School of Engineering, 2011 Annual Conference & Exposition, Vancouver, BC, (June, 2011)
Within the computer science arena, cloud computing has become a major topic of interest.Major computing corporations tout the advantages of such systems, including high reliability,scalability, remote availability, and other advantages. Major industries employ these types ofsystems in their daily business.In the academic arena, cloud computing usage, aside from basic e-mail and web hosting, hasbeen slower to develop. Some universities have outsourced email management and otheradministrative services. Outside of this area, the usage of cloud computing has generally beenlimited to individual faculty members using remote services. However, with the rapid escalationin cloud computing service availability, this is certain to change. This is not, however, withoutrisk, both at an institutional and individual level.This article addresses four aspects of cloud computing. First, it provides an overview of theservices. Two major types of systems will be profiled, including file synchronization systems(DropBox, Ubuntu-One, etc.) and project management systems (GForge). The article will thenaddress the impact upon an individual campus infrastructure. Beyond an IT aspect, the articlewill address the legal issues of using such a system, including the potential FERPA ramificationsto the institution and the faculty member. Lastly, a set of recommendations will be provided tofaculty members who are interested in using cloud functionality in their teaching work.
23. Practical Interpretation of Student Evaluations for Starting Professors
Walter W. Schilling, Milwaukee School of Engineering; John K. Estell, Ohio Northern University; Frederick Clayton Berry, Milwaukee School of Engineering, 2011 Annual Conference & Exposition, Vancouver, BC, (June, 2011)
Practical Interpretation of Student Evaluations for Starting ProfessorsAbstractIn nearly all university environments, students evaluate their instructors. However, howthese evaluations are used varies greatly from institution to institution and from positionto position. Compounding this fact is the different expectations of students at differentcampuses.As students and graduate students, the goal for a class is always an A. However, with studentevaluations, consistently obtaining perfect evaluations is nearly impossible, even for the mostveteran instructor. This article provides practical ideas for interpreting student evaluations. Itincludes a summary of how evaluations are used at teaching oriented universities at both adepartmental and college level. It then provides a set of tips for interpreting student evaluations,and provides techniques for setting improvement goals.
24. Using Performance Bonuses To Decrease Procrastination
Walter Schilling, MSOE, 2010 Annual Conference & Exposition, Louisville, Kentucky, (June, 2010)
A common problem amongst college students is procrastination. Students tend to wait until the last minute to work on projects, especially if the project is longer than a day or two in duration. By procrastinating, students often end up rushing at the end to complete the project, resulting in the student not obtaining the optimal educational benefit from the assignment. This syndrome also leads to long lines outside of the office door on the day or two before an assignment is due, as the students are frantically trying to complete an assignment and have questions that they cannot answer. Those who are able to work effectively under pressure complete the assignment, while those who do not either turn in substandard work or submit late assignments. The late assignments receive reduced credit and impact the delivery of the next assignment. These late assignments also defer grades being returned to other students, as it is difficult for fair and equitable grading to be ensured unless the majority of assignments are graded at the same time. In the construction industry, it is common for contractors to be assigned a performance bonus. Contracts that are finished early receive an incentive, typically in the form of an extra payment, while jobs which are finished late are penalized. In academics, it is common for a penalty to be assessed on late assignments. However, grading has not provided an incentive for students to finish their assignments early. This article analyzes the results of applying such an incentive system to engineering assignments. Through a small study, it will analyze whether performance bonuses reduce tardiness in assignment submissions as well as whether a performance bonus improves student learning through better achievement of course outcomes.
25. Using Your Grade Book To Store Course Rubric Information
Walter Schilling, MSOE, 2009 Annual Conference & Exposition, Austin, Texas, (June, 2009)
The usage of rubrics has been greatly shown to aid in consistent grading, faster grading, and improved student understanding of grades. Traditionally, the rubric is used to calculate a grade for the assignment, and then that final grade is recorded in the course grade book. This article presents a practical method for retaining within a grade book not only the final grade for an assignment, but also the individual sub-scores used to calculate that final grade. Grades are stored in Microsoft Excel, with each major assignment being stored on an individual workbook tab. Rows represent individual students, and the columns indicate the sub-scores for each line item on a grading rubric. Rubric scores are multiplied by a weighting factor to obtain an overall score for the assignment. From this data store, grade sheets, including grading comments, can automatically be generated for the students through a mail merge operation. These grade sheets can then either be printed in hard copy or e-mailed to the students as pdf attachments. Individual student performance can be tracked and graphed over time on multiple dimensions, allowing for easier observation of students who may be struggling with key concepts in the course, and the data can be used for overall assessment of course outcomes.
26. Tablet Pc Features On A Budget
Walter Schilling, MSOE, 2009 Annual Conference & Exposition, Austin, Texas, (June, 2009)
The usage of tablet PCs in engineering education represents a great step forward in improving classroom education. Numerous interactive demos can occur, and these activities can be recorded for student review. But this technology, while decreasing in price, can still be considered expensive if not made available to the faculty members by the Information Technology organization. Tablet PCs also suffer from reduced performance specifications versus comparable desktop or laptop models. However, even without the purchase of a tablet PC, certain benefits of the tablet PC system can be obtained through the usage of an inexpensive graphics tablet connected to the computer through a USB port. While typically used in graphic art environments, separate tablets can be used for interactive demos, assignment grading, and other activities which improve the classroom environment. This article explores the author’s experiences with such a system, whereby a separate graphics tablet is interfaced with a university issued laptop. Things gone wrong, as well as things gone right, with such a system will be developed.that final grade is recorded in the course grade book. This article presents a practical method for retaining within a grade book not only the final grade for an assignment, but also the individual sub-scores used to calculate that final grade. Grades are stored in Microsoft Excel, with each major assignment being stored on an individual workbook tab. Rows represent individual students, and the columns indicate the sub-scores for each line item on a grading rubric. Rubric scores are multiplied by a weighting factor to obtain an overall score for the assignment. From this data store, grade sheets, including grading comments, can automatically be generated for the students through a mail merge operation. These grade sheets can then either be printed in hard copy or e-mailed to the students as pdf attachments. Individual student performance can be tracked and graphed over time on multiple dimensions, allowing for easier observation of students who may be struggling with key concepts in the course, and the data can be used for overall assessment of course outcomes.
27. Issues Effecting Doctoral Students Returning To Engineering Education Following Extensive Industrial Experience
Walter Schilling, 2008 Annual Conference & Exposition, Pittsburgh, Pennsylvania, (June, 2008)
It can be said that the best time for an engineer who desires to enter academia to earn their Doctoral Degree is after a four to five year period working in industry. During this time period, a practicing engineering can be extensively exposed to the practical aspect of the engineering discipline. This exposure can aid to a more effective mentorship with students, better professional relationships with other practicing engineers, and a better understanding of the real problems faced by engineers solving real world problems. However, a practicing engineer who wishes to return for doctoral studies faces numerous hurdles not necessarily encountered by a traditional student. This article puts forth some of the challenges facing a practicing engineer who attempts to return to graduate school for Doctoral Study. Areas to be addressed include difficulties obtaining graduate admission, the lack of fellowships and grants for returning professionals, other financial aspects of graduate studies impacting practicing professionals, project management issues, and job search issues.
28. A methodology for quantitative evaluation of software reliability using static analysis
Walter W. Schilling, Jr. and Dr. Mansoor Alam, 2008 Annual Reliability and Maintainability Symposium, (January, 2008)
This paper proposes a methodology for quantitative evaluation of software reliability in updated COTS or Open Source components. The model combines static analysis of existing source code modules, limited testing with execution path capture, and a series of Bayesian Belief Networks. Static analysis is used to detect faults within the source code which may lead to failure. Code coverage is used to determine which paths within the source code are executed as well as their execution rate. A series of Bayesian Belief Networks is then used to combine these parameters and estimate the reliability for each method. A second series of Bayesian Belief Networks then combines the module reliabilities to estimate the net software reliability.
29. Evaluating the Effectiveness of Java Static Analysis Tools
Walter W. Schilling, Jr. and Dr. Mansoor Alam, 2007 International Conference on Embedded Systems and Applications (ESA 2007), (July, 2007)
Static analysis tools have been successfully used to aid in improving the reliability of embedded systems developed in the C and C++ programming languages. As embedded systems begin to be developed in the Java Programming language, it is important that Static Analysis tools be assessed for their effectiveness in discovering faults in Java source code. This article provides results on a controlled experiment using nine static analysis tools and the Java Programming language. The effectiveness of the tools at detecting faults is discussed, as is the correlation between the tools. The experiment shows that there is some overlap in fault detection between tools, but there also is an overlap between false positive detections as well. Overall, for the most effective results, it is recommended that multiple static analysis tools be used when analyzing a given software module.
30. Measuring the Reliability of Existing Web Servers
Walter W. Schilling, Jr. and Dr. Mansoor Alam, Proceedings of the 2007 IEEE International Electro/Information Technology Conference, (May, 2007)
As the usage of the Internet increases and more critical functions rely on the World Wide Web for operation, it is important that network researchers have current and accurateinformation from a real-world standpoint of the achieved reliability of the Internet. Anecdotal evidence suggests that web outages are a serious issue from a cost standpoint. However, there have been few mainstream studies showing the reliability of the existing web infrastructure. As the internet has changed, so to has its capabilities and its reliability. Yet, there have been few independent studies which have provided quantitative measurement of actual web reliability. This article provides results from a controlled experiment intended to measure the reliability of the existing web infrastructure.
31. Relating Software Reliability to Execution Rates Using Bayesian Belief Networks
Walter W. Schilling, Jr. and Dr. Mansoor Alam, NASA / Ohio Space Grant Consortium 2006-2007 Annual Student Research Symposium Proceedings XV, (May, 2007)
32. Integrate static analysis into a software development process
Walter W. Schilling, Jr. and Dr. Mansoor Alam,, Embedded Systems Design, (November, 2006)
Software reliability is an increasing risk to overall system reliability. As systems have grown larger and more complex, functionality in mission- and safety-critical systems is more often exclusively controlled through software. This change, coupled with improving reliability in hardware modules, has shifted the root cause of systems failure from hardware to software. Static analysis is a technique that can improve the quality and reliability of embedded systems software. Integrating static-analysis tools and techniques into the development process can yield significant reductions in development testing and field failures. However, integrating static analysis into a development process can be a daunting, especially if a large amount of legacy code is used in the development projects.
33. The Software Static Analysis Reliability Toolkit
Walter W. Schilling, Jr. and Dr. Mansoor Alam, ISSRE 2006: The 17th International Symposium on Software Reliability Engineering, (November, 2006)
A software reliability model combining static analysis, Bayesian Belief Networks, and code coverage measurement is proposed to estimate the reliability of existing software. Static analysis is used to detect faults within the source code which may lead to failure. Code coverage is used to determine which paths within the source code are executed as well as how often they execute. Finally, a Bayesian Belief Network is then used to combine these parameters and estimate the resulting software reliability. In order to facilitate the usage of this model, the Software Static Analysis Reliability Toolkit (SoSART) tool is proposed. This tool serves as both a reliability modeling tool and a bug finding meta tool suitable for comparing the results of different static analysis tools. This article discusses the purpose for this tool as well as the toolset capabilities.
34. Estimating Software Reliability with Static Analysis Techniques
Walter W. Schilling, Jr. and Dr. Mansoor Alam, 15th International Conference on Software Engineering and Data Engineering (SEDE-2006), (July, 2006)
Software unreliability represents an increasing risk to overall system reliability. Market forces, including decreased time to market, reduced development team sizes, and other factors, have encouraged projects to reuse existing software as well as to purchase COTS software solutions. This has made the usage of the more than 200 existing software reliability models increasingly dicult. It is imperative that practical reliability modeling techniques be developed to address these issues. It is on this premise that an appropriate software reliability model combining static analysis of existing source code modules, limited testing with path capture, and Bayesian Belief Networks is presented. Static analysis is used to detect faults within the source code which may lead to failure. Code coverage is used to determine which paths within the source code are executed as well as the execution rate. Finally, a Bayesian Belief Network is then used to combine these parameters and estimate the resulting software reliability.
35. Modeling the Reliability of Existing Software using Static Analysis
Walter W. Schilling, Jr. and Dr. Mansoor Alam, NASA / Ohio Space Grant Consortium 2005-2006 Annual Student Research Symposium Proceedings XIV, (May, 2006)
36. A Methodology for Estimating Software Reliability Using Limited Testing
Walter W. Schilling, Jr. and Dr. Mansoor Alam, Fastabstract: The 16th International Symposium on Software Reliability Engineering, (November, 2005)
As more advanced software systems have been developed incorporating reused source code and COTS components, the impact of software reliability has become increasingly important. Many mission critical and safety critical systems are now software controlled. As this transition has occurred, the root cause for systems failure has increasingly shifted from hardware to software. Traditional software reliability models require significant testing data to be collected during software development in order to estimate the reliability of the resultant software. However, if this data is not collected in a disciplined manner or is not made available to subsequent software engineers, these modeling techniques can not readily be applied. In this paper, we propose a different approach at estimating software reliability based upon limited testing, static analysis of the existing source code, and Bayesian belief networks.
37. WORK IN PROGRESS - MEASURING THE ROI TIME FOR STATIC ANALYSIS
Walter W. Schilling, Jr. and Dr. Mansoor Alam, 2005 Frontiers in Education Conference: Pedagogies and Technologies for the Emerging Global Economy, (October, 2005)
Static analysis is one method that offers potential for reducing errors in delivered software. Static analysis is currently used to discover buffer overflows and mathematical errors, as well as verifying compliance with documented programming standards. Static analysis is routinely used in safety critical software applications within the avionics and automotive industries. Outside of these applications, static analysis is not a routinely taught method for software development. This paper intends to provide a quantitative measure for evaluating the effectiveness of static analysis as well as presenting results from an academic environment.
38. Randomized playback of tracks in a multimedia player
Walter W. Schilling, Jr. and Jacob Tanana, United States Patent, (March, 2004)
An audio reproduction apparatus randomizes or shuffles the playback order of tracks from a prerecorded media such as a compact disc within a multiple-disc changer. Shuffle sequences are generated using a linear congruential random number generator (LCRNG). The shuffle sequence does not have to be stored since a next track in the sequence can always be generated from the LCRNG using a particular set of parameters. The parameters are determined in response to the number of track (or discs) in a sequence and include elements obtained from a lookup table and elements that are randomly selected so that the same sequence is not always generated for a certain sequence length.
39. The Impact of Prefetching and Victim Caching on Computer Systems Performance
Walter W. Schilling, Jr. and Dr. Mansoor Alam, Proceedings of the 12 International Conference on Parallel and Distributed Computing Systems, (August, 1999)